Trouble with “Plone [archetypes]” and “Products.ResourceRegistries” in 5.2.5

Plone Support
,
1

I have succeeded in migrating a site from 4.3.18 to 5.2.5 with Python2.7 but the 5.2.5 instance generates a bunch of DeprecationWarning messages that I think would be good to clean up if possible. The site is not complicated and was originally created in Plone about 20 years ago.

When I first migrated the site to 5.2.5, I was getting

AttributeError: type object 'IATCTTool' has no attribute 'iro'
AttributeError: type object 'IKSSRegistry' has no attribute 'iro'

error messages when running “sudo -u plone_daemon bin/instance fg”. I fixed these by updating eggs to include “Plone [archetypes]” and “Products.ResourceRegistries” and running buildout.

eggs =
Plone
Plone [archetypes]
Products.ResourceRegistries

With these eggs, the site loads fine but generates many DeprecationWarning messages.

root@plone2:/opt/plone5.2.5_python2.7.clean/zinstance# sudo -u plone_daemon bin/instance fg
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.Archetypes-1.16.5-py2.7.egg/Products/Archetypes/generator/widget.py:4: DeprecationWarning: InitializeClass is deprecated. Please import from AccessControl.class_init.
from App.class_init import InitializeClass
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.Archetypes-1.16.5-py2.7.egg/Products/Archetypes/ReferenceEngine.py:31: DeprecationWarning: LazyMap is deprecated. Please import from ZTUtils.Lazy.
from Products.ZCatalog.Lazy import LazyMap
2021-11-20 16:20:37,864 INFO [chameleon.config:38][MainThread] directory cache: /opt/plone5.2.5_python2.7.clean/zinstance/var/cache.
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/interfaces/factories.py:2: DeprecationWarning: IFileFactory is deprecated. Import IFileFactory from zope.filerepresentation.interfaces instead
from plone.app.widgets.interfaces import IFileFactory
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/permission.py:2: DeprecationWarning: IFieldPermissionChecker is deprecated. Import IFieldPermissionChecker from plone.app.z3cform.interfaces instead
from plone.app.widgets.interfaces import IFieldPermissionChecker
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/permission.py:8: DeprecationWarning: setDefaultRoles is deprecated. Please use addPermission from AccessControl.Permission.
from Products.CMFCore.permissions import setDefaultRoles
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/content/base.py:21: DeprecationWarning: ResourceLockedError is deprecated. Please import from zExceptions.
from webdav.Lockable import ResourceLockedError
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/lib/historyaware.py:4: DeprecationWarning: html_quote is deprecated. Please import from DocumentTemplate.html_quote. These shims will go away in DocumentTemplate 4.0.
from DocumentTemplate.DT_Util import html_quote
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/content/topic.py:35: DeprecationWarning: LazyCat is deprecated. Please import from ZTUtils.Lazy.
from Products.ZCatalog.Lazy import LazyCat
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/tool/factory.py:30: DeprecationWarning: call_object is deprecated. Please import from ZServer.ZPublisher.Publish.
from ZPublisher.Publish import call_object
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/tool/factory.py:31: DeprecationWarning: dont_publish_class is deprecated. Please import from ZServer.ZPublisher.Publish.
from ZPublisher.Publish import dont_publish_class
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/tool/factory.py:32: DeprecationWarning: missing_name is deprecated. Please import from ZServer.ZPublisher.Publish.
from ZPublisher.Publish import missing_name
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ExternalEditor-3.0.1-py2.7.egg/Products/ExternalEditor/ExternalEditor.py:23: DeprecationWarning: rfc1123_date is deprecated. Please import from zope.datetime. This backwards compatibility import will go away in Zope 6.
from App.Common import rfc1123_date
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ResourceRegistries-3.0.8-py2.7.egg/Products/ResourceRegistries/tools/BaseRegistry.py:40: DeprecationWarning: zope.site.hooks has moved to zope.component.hooks. Import of zope.site.hooks will become unsupported in Version 5.0
from zope.site.hooks import getSite
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/plone.app.upgrade-2.0.39-py2.7.egg/plone/app/upgrade/init.py:157: DeprecationWarning: GopipIndex is deprecated. Please import from plone.folder.nogopip
from plone.app.folder.nogopip import GopipIndex
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/plone.app.blob-1.8.1-py2.7.egg/plone/app/blob/content.py:23: DeprecationWarning: MimeTypeException is deprecated. Import from Products.MimetypesRegistry.interfaces instead
from Products.MimetypesRegistry.common import MimeTypeException
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/Products.ATContentTypes-3.0.4-py2.7.egg/Products/ATContentTypes/setuphandlers.py:9: DeprecationWarning: ILanguageSchema is deprecated. It has been moved to plone.i18n.interfaces, import from there instead.
from Products.CMFPlone.interfaces import ILanguageSchema
/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/plone.app.contenttypes-2.2.2-py2.7.egg/plone/app/contenttypes/migration/patches.py:13: DeprecationWarning: _marker is deprecated. Please import from Products.PluginIndexes.unindex.
from Products.PluginIndexes.common.UnIndex import _marker
2021-11-20 16:20:43,406 INFO [Zope:45][MainThread] Ready to handle requests
Starting server in PID 15970.
Serving on http://0.0.0.0:8080

Removing “Plone [archetypes]” and “Products.ResourceRegistries” from the eggs and running buildout fixes the above warning messages, but causes the two AttributeError messages to return and causes the site not to work.

I think it makes sense to somehow fix the site so that it does not depend on “Plone [archetypes]” and “Products.ResourceRegistries”.

I tried installing and removing Archetypes Content Types for Plone (backward compatibility) and Multilingual Support (Archetypes) from http://testing.losaltosrobotics.org:8080/Main/prefs_install_products_form but that did not work.

How can I remove the dependency of the site on “Plone [archetypes]” and “Products.ResourceRegistries”?

Perhaps these WARNING and ERROR messages from @@plone_upgrade will be helpful.

2021-11-20 16:43:21,214 WARNING [OFS.Uninstalled:84][waitress-1] Could not import class 'PasswordResetTool' from module 'Products.PasswordResetTool.PasswordResetTool'
2021-11-20 16:43:26,001 ERROR [GenericSetup:1296][waitress-2] Step languagetool has an invalid import handler
2021-11-20 16:43:26,328 WARNING [GenericSetup.portlets:100][waitress-2] Cannot register portlet type portlets.Calendar because it is already registered.
2021-11-20 16:43:26,329 WARNING [GenericSetup.portlets:100][waitress-2] Cannot register portlet type portlets.Events because it is already registered.
2021-11-20 16:43:30,105 WARNING [plone.app.upgrade:527][waitress-2] could not upgrade calendar_starting_year property
2021-11-20 16:43:30,105 WARNING [plone.app.upgrade:527][waitress-2] could not upgrade calendar_future_years_available property
2021-11-20 16:43:31,256 WARNING [GenericSetup.plone.app.registry:100][waitress-2] Interface Products.ResourceRegistries.interfaces.settings.IResourceRegistriesSettings specified for record %s has no field plone.lessvariables.
2021-11-20 16:43:32,707 WARNING [GenericSetup.componentregistry:100][waitress-2] The object None was not found, while trying to register an utility. The provided object definition was portal_syndication. The site used was: <PloneSite at /Main>
2021-11-20 16:43:32,707 WARNING [GenericSetup.componentregistry:100][waitress-2] The object None was not found, while trying to register an utility. The provided object definition was portal_undo. The site used was: <PloneSite at /Main>
2021-11-20 16:43:35,394 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile Products.CMFPlacefulWorkflow:CMFPlacefulWorkflow is unknown, refusing to upgrade.
2021-11-20 16:43:35,395 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile plone.app.contenttypes:default is unknown, refusing to upgrade.
2021-11-20 16:43:35,395 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile plone.app.dexterity:default is unknown, refusing to upgrade.
2021-11-20 16:43:35,454 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile plone.app.iterate:plone.app.iterate is unknown, refusing to upgrade.
2021-11-20 16:43:35,454 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile plone.app.multilingual:default is unknown, refusing to upgrade.
2021-11-20 16:43:35,456 WARNING [Products.GenericSetup.tool:1134][waitress-2] Version of profile plone.app.users:default is unknown, refusing to upgrade.

Unfortunately, the full output from @@plone_upgrade is too big to inlcude.

I would appreciate any help that you can give me with removing the site's dependencies on the “Plone [archetypes]” and Products.ResourceRegistries eggs. Thanks, Michael

2

Archetypes and ATContentTypes are no longer maintained; migrate to dexterity. Plone no longer uses ResourceRegistries but instead uses the new (general purpose) registry. There are no Python 3 compatible versions for those packages - you should get rid of them in your installation.

3

@dieter Thank you very much for telling me to migrate to dexterity. Somehow I had missed understanding this requirement. I used the dexterity upgrade tool for my site and my site looks like it is using dexterity.

My site still fails to load if it is built without the Plone [archetypes] and Products.ResourceRegistries eggs. I get the

AttributeError: type object 'IATCTTool' has no attribute 'iro '
AttributeError: type object 'IKSSRegistry' has no attribute 'iro '

error messages if these eggs are not included.

It appears that I am missing a step somewhere. Is there something else that I need to do to get rid of ResourceRegistries?

I don't know if it is related or not, but after upgrading to Plone 5.2.5, I was getting a 12 DirectoryView XX refers to a non-existing path warning messages. After some hunting around, I found a bunch of skins that are not in a clean install of Plone 5.2.5. I found them in portal_skins in the zope management interface.

I deleted:

  • archetypes
  • ATContentTypes
  • classic_ecmascript
  • classic_styles
  • cmfeditions_views
  • cmf_legacy
  • LanguageTool
  • mimetypes_icons
  • PasswordReset
  • plone_3rdParty
  • plone_content
  • plone_deprecated
  • plone_forms
  • plone_styles
  • referencebrowser
  • sunburst_images
  • sunburst_styles
  • sunburst_templates
  • tinymce

and kept:

  • CMFEditions
  • custom
  • plone_form_scripts
  • plone_images
  • plone_prefs
  • plone_scripts
  • plone_templates
  • plone_wysiwyg

After deleting these, the site loads without generating any DirectoryView XX refers to a non-existing path warning messages.

Is there something like this that I need to do to remove all dependencies on the Plone [archetypes] and Products.ResourceRegistries eggs?

4

You must delete the Archetypes and ResourceRegistries related objects -- before you remove the corresponding code. Otherwise, you still have the old objects in the ZODB but the necessary code is no longer available. This will lead to "broken" objects which behave wrongly.

5

Yes.
For a client, I used:

def process_portal(p, options):
  """detect and potentially remove AT objects."""
  # process objects
  i = p.portal_catalog._catalog.indexes["object_provides"]
  ks = list(i._index.keys("Products.ATContentTypes.", "Products.ATContentTypes.\xff"))
  if ks:
    at_pxs = p.portal_catalog.unrestrictedSearchResults(object_provides=ks)
    if at_pxs:
      pp_len = len(p.getPhysicalPath())
      at_paths = [px.getPath().split("/")[pp_len:] for px in at_pxs]
      if options.remove:
        deleted = []
        # delete from bottom upwards
        at_paths = sorted(at_paths, key=lambda x: len(x), reverse=True)
        for pa in at_paths:
          __traceback_info__ = pa
          obj = p.unrestrictedTraverse(pa)
          obj.aq_parent._delObject(obj.getId())
          deleted.append("/".join(pa))
        logger.info("deleted objects %s", deleted)
      else:
        logger.info("found objects %s", ["/".join(pa) for pa in at_paths])
  # delete `portal_languages` "beforeTraverse"
  from ZPublisher.BeforeTraverse import unregisterBeforeTraverse
  r = unregisterBeforeTraverse(p, "Plone Language Tool/portal_languages")
  if r: logger.info("unregistered `portal_languages` beforeTraverse")
  # delete tools
  tools = [t.getId() for t in p.objectValues() if t.__class__.__module__.startswith("Products.Archetypes") or t.__class__.__module__.startswith("Products.ATContentTypes")]
  if "portal_languages" in p.objectIds():
    tools.append("portal_languages")
  if tools:
    for t in tools: p._delObject(t)
    logger.info("deleted %s", tools)
  # work around `ATCTTool` not cleaning up the "local component registry"
  from Products.ATContentTypes.interfaces.interfaces import IATCTTool
  p._components.unregisterUtility(provided=IATCTTool)

In your case, not everything might be necessary (e.g. the deletion of remaining AT objects). On the other hand, you might need things not necessary for my client.

6

Thank @dieter for the guidance and code. I am still coming up to speed on how to run code like you have given me. I copied your code into dieter.py and ran it with:

cd zinstance
cat > dieter.py
sudo -u plone_daemon bin/instance run dieter.py

While dieter.py ran without generating any errors, it did not appear to do anything. I have two sites named Main and SchuhTest. I looked in the ZMI in /manage_main and /Main/manage_main and did not find process_portal.

How do I run your code?

After going through your code, I looked at ZMI portal_catalog and compared my /Main site to a site in a clean installation of Plone 5.2.5 but did not find any differences.

I went to /Main/portal_quickinstaller/manage_installProductsForm and saw that my site had more products installed than a clean install of Plone 5.2.5. It also had older "Version at Install time" versions than the "Product version" for a number of products. I tried to delete the extra ones in my /Main site and reinstall some of the others, but this was not enough to remove the dependence on the Plone [archetypes] and Products.ResourceRegistries eggs. Here is what I currently have in /Main/portal_quickinstaller/manage_installProductsForm

> Installed Products
> Product	Version at Install time	Product version
> 	Barceloneta Theme	2.1.10	2.1.10
> 	CMFEditions	3.3.4	3.3.4
> CMFFormController	1.0.4	4.1.4
> Collection portlet	1.1.3	3.3.6
> 	Configuration registry	1.7.8	1.7.8
> 	Content-types (plone.app.contenttypes)	2.2.2	2.2.2
> 	Dexterity Content Types	2.6.9	2.6.9
> 	Dexterity versioning support	1.4.3	1.4.3
> 	Diazo theme support	4.1.7	4.1.7
> 	HTML Output Filters	4.0.2	4.0.2
> 	HTTP caching support	2.1.0	2.1.0
> 	Import User Data	2.6.6	2.6.6
> 	MimetypesRegistry	2.1.8	2.1.8
> 	Plone z3c.form support	3.2.2	3.2.2
> 	PlonePAS	6.0.8	6.0.8
> 	PortalTransforms	3.1.11	3.1.11
> 	Querystring parser/builder	1.4.14	1.4.14
> 	Relation Field	2.0.3	2.0.3
> 	Static resource storage	2.1.4	2.1.4
> Static text portlet	1.2rc1	3.1.6
> 	plone.app.event	3.2.12	3.2.12
> 	plone.app.intid: install utility	1.1.4	1.1.4
> 	plone.app.linkintegrity	3.4.1	3.4.1
> 	plone.formwidget.recurrence	2.1.4	2.1.4
> 	plone.staticresources EXPERIMENTAL: Async Resource Loading	1.4.3	1.4.3

I am unable to upgrade CMFFormController, Collection portlet, and Static text portlet because there is not a Reinstall selector box next to them. I tried to Reinstall Mandatory dependencies for a Plone site but that failed due to a "Forbidden: Form authenticator is invalid" error.

I would appreciate any guidance that you give me. Thanks.

7

Sure. I had hoped that my remark at the end of my post would encourage you to carefully look at the code (and not just run it). This would have shown you that my code defines a function; it must be called to get a (visible) effect. In addition, it does nothing with transactions; therefore, after a successful run, the transaction needs to be committed.

My client has a huge number of portals; therefore, there is general infrastructure to run a function for many/all portals. This infrastructure handles all general aspects such as argument processing, transaction control, error handling, logging configuration ... The function I have posted targets this infrastructure. You cannot simple run it.