Upgrade to 5.2 failing with 'IATCTTool' has no attribute '__iro__'

Plone Support
, ,
21

I migrated from 4.3 to 5.1 and then to 5.2-python2. I had to delete several product objects that didn't have support for Plone 5.1 and 5.2 at the time (collective.cover, PloneFormGen, Plone True Gallery, Reset Password Tool), I also used wildcard.fixpersistentutilities and @atct_migrator. I had to add wiked and Plone [archetypes] in buildout.cfg in version 5.2-python2. I had to uninstall Products.Archetypes and plone.app.collections for each site using portal_quickinstaler, in addition to deleting the folders portal_atct, portal_factory, portal_languages, and portal_tinymce.

So far everything has worked on Plone 5.2-python2, but I haven't been able to migrate to version 5.2-python3. I was able to run zodbupdate after adding wicked, Plone [archetypes] and eea.aliases to [buildout] eggs, but when I start Plone 5.2-python3 there are several errors like "AttributeError: type object 'IATCTTool' has no attribute 'iro' ".

I returned to Plone 5.2-python2 and followed the procedure described by fulv, getting the following error messages:

  • pleonformgen - ERROR: Step pleonformgen has an invalid import handler
    toolset: Toolset imported.
  • languagetool - ERROR: Step languagetool has an invalid import handler
    toolset: Toolset imported.
  • collective.plonetruegallery.install - ERROR: Step collective.plonetruegallery.install has an invalid import handler
    collective.plonetruegallery.uninstall - ERROR: Step collective.plonetruegallery.uninstall has an invalid import handler
  • collective.portlet.calendar-upgrades - ERROR: Step collective.portlet.calendar-upgrades has an invalid import handler
  • collective.z3cform.datetimewidget - ERROR: Step collective.z3cform.datetimewidget has an invalid import handler

When I run zodbupdate and start Plone 5.2-python 3 there are still errors (AttributeError: type object 'IATCTTool' has no attribute 'iro' ) and Plone does not work.

Can someone help me?

How do I execute the script reported by @pbauer (Aug '19)? Do I save it to a clean_plone.py file and run "bin/instance run /plone/instance/clean_plone.py"?

This did not work for me. I got the following error messages:

Traceback (most recent call last):
    File "/plone/instance/parts/instance/bin/interpreter", line 287, in <module>
        exec(_val)
    File "<string>", line 1, in <module>
    File "/plone/instance/clean_plone.py", line 15, in <module>
        portal.manage_delObjects([tool])
NameError: name 'portal' is not defined
22

All the code in https://github.com/collective/collective.migrationhelpers is meant to be used in your own upgrade-steps. If you don't know what that means I suggest you read https://training.plone.org/5/mastering-plone/dexterity_2.html#upgrade-steps and https://docs.plone.org/develop/addons/components/genericsetup.html#upgrade-steps.

Simple examples how they are used in real life for upgrade-steps that migrated https://www.plone.de from Plone 4.3 to 5.2 can be found in https://github.com/collective/plonesite.de/blob/master/src/plonede.policy/plonede/policy/upgrades.zcml and https://github.com/collective/plonesite.de/blob/master/src/plonede.policy/plonede/policy/upgrades.py.

More examples are mentioned in Small project migration with customized barceloneta.

23

Errors like this typically indicate that the definition of the reference interface is not available. I would not be surprised as ATCT refers to "ArcheType based Content Types" and as far as I know, there is not Python 3 compatible Archetypes version. Eliminate Archetypes (migrating to dexterity) before you switch to Python 3.

24

@@atct_migrator shows that there is no content to be migrated for all sites in Plone 5.2-python2.

25

I think atct_migrator only shows the default content types.

You could try /@@custom_migration and see if there is something there.

26

/@@custom_migration also shows that there is no content to migrate.

27

Thank you, @pbauer.

I'm trying to use https://github.com/collective/collective.migrationhelpers to clean up the sites. I ran a few things on Plone 5.2-python2 to try to clean up, but when I run zodbverify the following errors still appear:

2020-02-20 14:59:33,844 INFO    [zodbverify:64][MainThread]                                                                                                                                                                                                                    
Could not process unknown record '\x00\x00\x00\x00\x00\xa3sP':                                                                                                                                                                                                                 
2020-02-20 14:59:33,844 INFO    [zodbverify:65][MainThread] '\x80\x02cProducts.ResourceRegistries.tools.CSSRegistry\nStylesheet\nq\x01.\x80\x02}q\x02(U\x05_dataq\x03U\x08\x00\x00\x00\x00\x00\xa3sQq\x04cPersistence.mapping\nPersistentMapping\nq\x05\x86QU\nisExternalq\x06\x89u.'                                                                                                                                                                                                                                                                         
2020-02-20 14:59:33,844 INFO    [zodbverify:66][MainThread] Traceback (most recent call last):                                                                                                                                                                                 
File "/plone/buildout-cache/eggs/zodbverify-1.0.1-py2.7.egg/zodbverify/verify.py", line 58, in verify_record                                                                                                                                                                 
    class_info = unpickler.load()                                                                                                                                                                                                                                              
AttributeError: 'module' object has no attribute 'Stylesheet'                                                                                                                                                                                                                  
                                                                                                                                                                                                                                                            
2020-02-20 14:59:35,259 INFO    [zodbverify:48][MainThread] Done! Scanned 2533605 records.                                                                                                                                                                                     
Found 1047 records that could not be loaded.                                                                                                                                                                                                                                   
Exceptions and how often they happened: 
ImportError: No module named controlpanel.markup: 5
ImportError: No module named blocks.interfaces: 5
ImportError: No module named Product: 2
ImportError: No module named tiles.modifiers: 5
AttributeError: 'module' object has no attribute 'Stylesheet': 307
AttributeError: 'module' object has no attribute 'JavaScript': 428
ImportError: No module named piwik.core.interfaces: 3
ImportError: No module named js.jqueryui.controlpanel: 288
ImportError: No module named cover.content: 4

Can I remove these 1047 records in Plone 5.2-python2? How can I do this?

28

Uninstalling Products.Archetypes and plone.app.collections for each site using portal_quickinstaler again, in addition to deleting the folders 'portal_languages', 'portal_tinymce', 'kupu_library_tool', 'kupu_library_tool', 'portal_factory', 'portal_atct', 'uid_cct', 'uid_cctatal' , 'archetype_tool', 'reference_catalog' and 'portal_metadata', the following errors remain:

2020-02-21 12:49:48,330 INFO    [zodbverify:64][MainThread] 
Could not process <class 'plone.registry.field.Bool'> record '\x00\x00\x00\x00\x00\x81\x07\xee':
2020-02-21 12:49:48,330 INFO    [zodbverify:65][MainThread] 'cplone.registry.field\nBool\nq\x01.}q\x02(U\x17_Element__tagged_valuesq\x03}q\x04U\x0bdescriptionq\x05X\x00\x00\x00\x00U\x05titleq\x06X\x05\x00\x00\x00Mouseq\x07U\x07defaultq\x08I00\nU\x08requiredq\tI00\nU\x08readonlyq\nI00\nU\tfieldNameq\x0bU\x08ui_mouseq\x0cU\x08__name__q\rU\x05valueq\x0eU\tinterfaceq\x0fccollective.js.jqueryui.controlpanel\nIJQueryUIPlugins\nq\x10U\x0edefaultFactoryq\x11NU\x07__doc__q\x12h\x07U\rinterfaceNameq\x13U4collective.js.jqueryui.controlpanel.IJQueryUIPluginsq\x14u.'
2020-02-21 12:49:48,330 INFO    [zodbverify:66][MainThread] Traceback (most recent call last):
File "/plone/buildout-cache/eggs/zodbverify-1.0.1-py2.7.egg/zodbverify/verify.py", line 60, in verify_record
    unpickler.load()
ImportError: No module named js.jqueryui.controlpanel

2020-02-21 12:50:55,574 INFO    [zodbverify:48][MainThread] Done! Scanned 2412342 records. 
Found 310 records that could not be loaded. 
Exceptions and how often they happened: 
ImportError: No module named controlpanel.markup: 5
ImportError: No module named blocks.interfaces: 5
ImportError: No module named js.jqueryui.controlpanel: 288
ImportError: No module named tiles.modifiers: 5
ImportError: No module named piwik.core.interfaces: 3
ImportError: No module named cover.content: 4
29

After several attempts, I migrated the content of 8 plone sites (38GB) from 5.2-python2 to 5.2-python3. I had already managed to migrate from 4.3 to 5.2-python2, but not to 5.2-python3. This may not be the smartest and recommended way, but it worked.

For each of the sites (MySiteA, MySiteB, MySiteC, ...), I followed the following procedures:

  1. I created a new MySiteA2 website in Plone 5.2-python2;
  2. I cut out the content of the MySiteA website; (http: //my.plone.server/mysitea/folder_contents) and pasted it on the MySiteA2 website (http://my.plone.server/mysitea2/folder_contents);
  3. I exported the plone site mysitea2 using the "Import / Export" option in the Zope management interface (http://my.plone.server/manage_main).

After generating the .zexp file for all sites I did the following:
4) I deleted the entire volume of data from Plone 5.2-python2 (I use docker), that is, I started a completely clean plone;
5) I copied the .zexp files from each of the sites to the import folder (/data/instance/import, for my container);
5) I imported each of the sites using the "Import/Export" option in the Zope management interface (http://my.plone.server/manage_main), using the standard options;

Finally, I deleted Data.fs.index and ran zodbupdate (zodbupdate --convert-py3 --file=var/filestorage/ Data.fs --encoding = utf8 --encoding-fallback latin1).

When I started Plone 5.2-python3 some errors occurred due to the sharing of some folders and pages. To solve this problem, I installed, uninstalled and reinstalled the pas.ldap.plugin add-on.

Another thing I needed to do was add the Portuguese language, in addition to Portuguese (Brazil) at http://srv-ldock-009.tre-rn.jus.br:8888/biometria/@@language-controlpanel.

The side effect that I've identified so far has been losing all sharing settings for folders, folders, etc.

I believe these steps have worked for cleaning up broken objects and configurations that have existed on my plone sites since Plone 4.3.

1 Like
30

Be careful using the zexp export/import. The imported objects will have new intids and this will break relations and possibly make it difficult to create new relations with them (for things like Related Items). See Maintaining zope intids on zexp imports. I don't even bother trying to use the zexp export/import anymore and think it should probably be dropped in modern Plone. It's almost always better to copy an entire database or serialize/deserialize in some other way.

1 Like
31

Hi Everyone,

I have been working on upgrading my site from 5.0.6 to 5.2.x in the hope of eventually getting it to 6.0. In the last few days I have spent a lot of time getting it from 5.0.6 -> 5.1.10 -> 5.2.10 (python2) -> dexterity upgrade -> 5.2.10 (python 3 + db upgrade).

I had run into the issue in this thread, and followed the steps the script above and was able to get the site over to python 3 and everything view wise seems to be working.

However, I have noticed after when I tried to edit a page or add a new item I am getting this portal_languages error.

We’re sorry, but there seems to be an error…
Here is the full error message:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 162, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 371, in publish_module
  Module ZPublisher.WSGIPublisher, line 266, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 63, in call_object
  Module plone.z3cform.layout, line 63, in __call__
  Module plone.z3cform.layout, line 47, in update
  Module plone.dexterity.browser.add, line 138, in update
  Module plone.z3cform.fieldsets.extensible, line 65, in update
  Module plone.z3cform.patch, line 30, in GroupForm_update
  Module z3c.form.group, line 141, in update
  Module z3c.form.group, line 52, in update
  Module z3c.form.group, line 48, in updateWidgets
  Module z3c.form.field, line 277, in update
  Module z3c.form.browser.select, line 51, in update
  Module z3c.form.browser.widget, line 171, in update
  Module z3c.form.widget, line 234, in update
  Module Products.CMFPlone.patches.z3c_form, line 47, in _wrapped
  Module z3c.form.widget, line 115, in update
  Module zope.schema._bootstrapfields, line 109, in __get__
  Module plone.app.dexterity.behaviors.metadata, line 58, in default_language
  Module Products.CMFCore.utils, line 13, in check_getToolByName
  Module Products.CMFCore.utils, line 139, in getToolByName
AttributeError: portal_languages

Is this related to me removing portal_langauges in the steps I followed above? Is there any way to fix this?

32

Andrew Calcutt via Plone Community wrote at 2023-4-3 22:21 +0000:

...
Error messages of the form
IXXX has no attribute __iro__
indicate that the code for IXXX is missing.

Install Products.ATContentTypes (to get the code for
IATCTool back; then delete all tools related to Archetypes.
After that, you should be able to uninstall Products.ATContentTypes again.

33

That has to be done back on the python 2.7 version right? And by Install you mean add "Products.ATContentTypes" to my list of eggs and run buildout? I had tried to add it on the python 3 version and it gets an error.

On my 5.2 python 2.7 instance, I had already gone through and deleted all the tools related to Archetypes (listed in post 17 script) and ran 'profile-Products.ATContentTypes:uninstall', so I don't really notice a difference with or without archtypes or Products.ATContentTypes installed ("IXXX has no attribute iro" does not come back). I notice editing pages and adding new pages is working in this python 2.7 version of 5.2 though.

I do likely have a few uncleanly uninstalled plugins, but as far as I can tell none of them are installed anymore. This site been running since around 2007, so it has been though many upgrades and plugins that it has outlived, which I see a few of in zodbverify. like
quickupload.portlet.quickuploadportlet
openid.plugins.oid
recaptcha.settings

What's the best way to find tools related to Archetypes an uninstall them?

34

Andrew Calcutt via Plone Community wrote at 2023-4-4 00:31 +0000:

That has to be done back on the python 2.7 version right?

After your AT -> dexterity conversion.

At this point, you still have Products.ATContentTypes installed.
You can then delete the AT related tools directly.

And by Install you mean add "Products.ATContentTypes" to my list of eggs and run buildout? I had tried to add it on the python 3 version and it gets an error.

Indeed.

On my 5.2 python 2.7 instance, I had already gone through and deleted all the tools related to Archetypes (listed in post 17 script) and ran 'profile-Products.ATContentTypes:uninstall', so I don't really notice a difference with or without archtypes or Products.ATContentTypes installed ("IXXX has no attribute iro" does not come back). I notice editing pages and adding new pages is working in this python 2.7 version of 5.2 though.

That you do see the IATCT object has no __iro__ error
means that your ZODB still references the IATCT interface.
Either, you have forgotten to delete some Archetypes related tools
or (due to some bug) some tool deletion forgot to unregister
the tool with the local component registry.

You might try the extension wildcard.fixpersistentregistry
(or similarly named). It exists to work around
and partially fix situations like you observe one.

...
What's the best way to find tools related to Archetypes an uninstall them?

When I speak of "tool", I mean "tool in the sense of CMFCore,
i.e. a Zope object residing in the portal root and performing
some function. Often, such tools have a name of the form portal_XXXX,
e.g. portal_url.

Those tools are not uninstalled but deleted (via the ZMI).

Ideally, the uninstall of the corresponding package
(e.g. via portal_quickinstaller) would also delete the
corresponding tools. E.g. the uninstall of Products.ATContentTypes
and Products.Archetypes should perform the tool deletion
automatically.
BUT the uninstall is very often incomplete.
When I remember right, I had to delete some Archetypes related tools manually,
when I migrated to dexterity.

To find potentially unnecessary tools, you could try the following
approach:
Build a new Plone portal with all extensions you would like to use
(the extensions may add tools).
Compare the objects in this new portal with those of your
migrated portal: objects in the migrated portal not present
in the new portal are potential candidates for deletion.

35

IIRC correctly you just delete the portal_languages tool from the ZMI and live happy with that.
Be sure that your language settings are fine,

BTW the code that should have removed the portal_languages tool is this one:

36

Thanks everyone for the tips. I was able to get my site up to 5.2.11(python 3) with the tips in this thread. I'm honesty not sure what I did differently. I went back to 5.1.10 and ran wildcard.fixpersistentregistry there and tried to clean up a few things, but there wasn't a lot there to clean up. I then did the upgrade again to 5.2.10 py2 and applied the fix in this thread. once i did that I upgraded to 5.2.10 py3 and this time things seem to work properly.

I did run into one issue with this Editing items with very old-style DiscussionItem breaks · Issue #2877 · plone/Products.CMFPlone · GitHub , but was able to at least work around it by turning discussion and version history off on the affected pages.

37

Just a note: there's also the great package GitHub - collective/collective.migrationhelpers: Some helpers and examples to use during migrations which has scripts and ideas how to handle leftovers in a inplace migration.

38

We've written about this over at Removing a persistent local utility part II - Four Digits too. The section 'sitemanager' show how to remove stuff when the interfaces classes are already removed from your environment.

If this is already somewhere in the docs, please link to it :slight_smile: