Can't pickle class 'plone.app.upgrade.bbb.ITinyMCE'

acalcutt · April 6, 2023, 10:12pm

When trying to upgrade my sites from 5.2.11(py3) to 6.0.3 I am getting this error

_pickle.PicklingError: Can't pickle <class 'plone.app.upgrade.bbb.ITinyMCE'>: import of module 'plone.app.upgrade.bbb' failed
End of upgrade path, main migration has finished.

I haven't found anything on how to fix this error...the closest I have found is bbb could posibly relate to something done for backward compatibility. What does this "BBB" mean, actually?

Does anyone have any suggestions on how to fix this error. this is the full upgrade log

Upgrade report
Starting the migration from version: 5219
Role / permission map imported.
Types tool imported.
'Plone Site' type info imported.
Ran upgrade step: Be sure that the Plone Site FTI is a dexterity one
Migrating object 'portal_setup'
Migrating object 'caching_policy_manager'
Migrating object 'content_type_registry'
Migrating object 'error_log'
Migrating object 'plone_utils'
Migrating object 'portal_actions'
Migrating object 'portal_catalog'
Migrating object 'portal_controlpanel'
Migrating object 'portal_diff'
Migrating object 'portal_groupdata'
Migrating object 'portal_groups'
Migrating object 'portal_memberdata'
Migrating object 'portal_membership'
Migrating object 'portal_migration'
Migrating object 'portal_properties'
Migrating object 'portal_quickinstaller'
Migrating object 'portal_registration'
Migrating object 'portal_skins'
Migrating object 'portal_types'
Migrating object 'portal_uidannotation'
Migrating object 'portal_uidgenerator'
Migrating object 'portal_url'
Migrating object 'portal_view_customizations'
Migrating object 'portal_workflow'
Migrating object 'translation_service'
Migrating object 'portal_form_controller'
Migrating object 'mimetypes_registry'
Migrating object 'portal_transforms'
Migrating object 'acl_users'
Migrating object 'portal_archivist'
Migrating object 'portal_historiesstorage'
Migrating object 'portal_modifier'
Migrating object 'portal_purgepolicy'
Migrating object 'portal_referencefactories'
Migrating object 'portal_repository'
Migrating object 'portal_uidhandler'
Migrating object 'HTTPCache'
Migrating object 'RAMCache'
Migrating object 'ResourceRegistryCache'
Migrating object 'P1010002.JPG'
Migrating object 'apache_pb2.GIF'
Migrating object 'apache_pb2_ani.gif'
Migrating object 'awstats'
Migrating object 'contact-us'
Migrating object 'cs-game-server'
Migrating object 'copy_of_front-page'
Migrating object 'header.jpg'
Migrating object 'hl2dmlogo.jpg'
Migrating object 'penglogo_thumb.jpg'
Migrating object 'powered_by_win2k3.gif'
Migrating object 'copy_of_robots.txt'
Migrating object 'srvers.JPG'
Migrating object 'untitled.bmp'
Migrating object 'wrouter.JPG'
Migrating object 'pics-1'
Migrating object 'MailHost'
Migrating object 'portal_historyidhandler'
Migrating object 'MARIO_KART.bsp'
Migrating object 'portal_registry'
Migrating object 'tech-docs'
Migrating object 'portal_resources'
Migrating object 'portal_password_reset'
Migrating object 'portal_languages'
Migrating object 'portal_tinymce'
Ran upgrade step: Make the Plone Site a dexterity container
Role / permission map imported.
Not creating required tool portal_languages, because class Products.PloneLanguageTool.LanguageTool is not found.
Not creating required tool portal_tinymce, because class Products.TinyMCE.utility.TinyMCE is not found.
Toolset imported.
Actions tool imported.
Control panel imported.
Ran upgrade step: Run to6000 upgrade profile.
Removed broken temp_folder from Zope root.
Removed temp_folder from Zope root _mount_points.
Ran upgrade step: Remove broken temp_folder / tempstorage / Products.TemporaryStorage
Ran upgrade step: Fix UUID for DX Site Root
Ran upgrade step: Index the Site Root
Ran upgrade step: Fix unicode properties
Role / permission map imported.
Control panel imported.
Ran upgrade step: Run to6003 upgrade profile.
Role / permission map imported.
Control panel imported.
Skins tool imported.
'Event' type info imported.
Imported.
Ran upgrade step: Run to6004 upgrade profile.
Role / permission map imported.
Removed record plone.bundles/resourceregistryt.jscompilation.
Removed record plone.bundles/resourceregistryt.csscompilation.
Removed record plone.bundles/resourceregistryt.expression.
Removed record plone.bundles/resourceregistryt.enabled.
Removed record plone.bundles/resourceregistryt.depends.
Removed record plone.bundles/resourceregistryt.load_async.
Removed record plone.bundles/resourceregistryt.load_defer.
Removed record plone.bundles/resourceregistryt.compile.
Removed record plone.bundles/resourceregistryt.resources.
Removed record plone.bundles/resourceregistryt.last_compilation.
Removed record plone.bundles/resourceregistryt.develop_javascript.
Removed record plone.bundles/resourceregistryt.develop_css.
Removed record plone.bundles/resourceregistryt.stub_js_modules.
Removed record plone.bundles/resourceregistryt.merge_with.
Role / permission map imported.
Role / permission map imported.
Role / permission map imported.
Role / permission map imported.
Role / permission map imported.
Role / permission map imported.
Role / permission map imported.
Removed 1226 records from registry
Removed bundle filemanager
Removed bundle plone-base
Removed bundle plone-datatables
Removed bundle plone-editor-tools
Removed bundle plone-fontello
Removed bundle plone-glyphicons
Removed bundle plone-moment
Removed bundle plone-tinymce
Removed bundle resourceregistry
Removed bundle thememapper
Removed bundle plone-legacy
Removed bundle plone-logged-in
Removed 29 deprecated bundle attributes from registry
Control panel imported.
Control panel imported.
Control panel imported.
Control panel imported.
Ran upgrade step: Cleanup resources and bundles
Added image scale: huge 1600:65536
Added image scale: great 1200:65536
Added image scale: larger 1000:65536
Added image scale: teaser 600:65536
Ran upgrade step: Add new image scales.
Role / permission map imported.
Skins tool imported.
Ran upgrade step: Run to6005 upgrade profile.
Added image_scales column to catalog metadata schema.
Updating metadata.
Upgrade aborted. Error:
Traceback (most recent call last):
  File "/opt/Plone_6/eggs/Products.CMFPlone-6.0.3-py3.8.egg/Products/CMFPlone/MigrationTool.py", line 299, in upgrade
    step['step'].doStep(setup)
  File "/opt/Plone_6/eggs/Products.GenericSetup-3.0.1-py3.8.egg/Products/GenericSetup/upgrade.py", line 193, in doStep
    self.handler(tool)
  File "/opt/Plone_6/eggs/plone.app.upgrade-3.0.3-py3.8.egg/plone/app/upgrade/v60/alphas.py", line 377, in update_catalog_for_image_scales
    update_catalog_metadata(context, column=column)
  File "/opt/Plone_6/eggs/plone.app.upgrade-3.0.3-py3.8.egg/plone/app/upgrade/utils.py", line 393, in update_catalog_metadata
    pghandler.report(index)
  File "/opt/Plone_6/eggs/Products.ZCatalog-7.0-py3.8.egg/Products/ZCatalog/ProgressHandler.py", line 55, in report
    transaction.savepoint(optimistic=True)
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_manager.py", line 272, in savepoint
    return self.manager.savepoint(optimistic)
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_manager.py", line 150, in savepoint
    return self.get().savepoint(optimistic)
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_transaction.py", line 228, in savepoint
    self._saveAndRaiseCommitishError()  # reraises!
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_transaction.py", line 316, in _saveAndRaiseCommitishError
    reraise(t, v, tb)
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_compat.py", line 49, in reraise
    raise value
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_transaction.py", line 225, in savepoint
    savepoint = Savepoint(self, optimistic, *self._resources)
  File "/opt/Plone_6/eggs/transaction-3.0.1-py3.8.egg/transaction/_transaction.py", line 626, in __init__
    savepoint = savepoint()
  File "/opt/Plone_6/eggs/ZODB-5.8.0-py3.8.egg/ZODB/Connection.py", line 994, in savepoint
    self._commit(None)
  File "/opt/Plone_6/eggs/ZODB-5.8.0-py3.8.egg/ZODB/Connection.py", line 545, in _commit
    self._store_objects(ObjectWriter(obj), transaction)
  File "/opt/Plone_6/eggs/ZODB-5.8.0-py3.8.egg/ZODB/Connection.py", line 576, in _store_objects
    p = writer.serialize(obj)  # This calls __getstate__ of obj
  File "/opt/Plone_6/eggs/ZODB-5.8.0-py3.8.egg/ZODB/serialize.py", line 434, in serialize
    return self._dump(meta, obj.__getstate__())
  File "/opt/Plone_6/eggs/ZODB-5.8.0-py3.8.egg/ZODB/serialize.py", line 443, in _dump
    self._p.dump(state)
_pickle.PicklingError: Can't pickle <class 'plone.app.upgrade.bbb.ITinyMCE'>: import of module 'plone.app.upgrade.bbb' failed
End of upgrade path, main migration has finished.
The upgrade path did NOT reach current version.
Migration has failed

davisagli · April 8, 2023, 6:52pm

Something in your site has a reference to plone.app.upgrade.bbb.ITinyMCE, but it was already removed from plone.app.upgrade. It was removed because the site should already have no references to it by the time it was upgraded to 5.2. You'll need to fix this in Plone 5.2 before you can run the upgrade to Plone 6. The reference is probably in the portal_tinymce tool or its registration as a utility.

acalcutt · April 8, 2023, 7:51pm

I had deleted portal_tinymce in the py2 to py3 migration steps in the fixes for https://community.plone.org/t/upgrade-to-5-2-failing-with-iatcttool-has-no-attribute-iro

How would I clean up its registration as a utility in 5.2?

davisagli · April 8, 2023, 8:55pm

It should be something like this:

Run bin/instance debug
sm = app.Plone.getSiteManager()
from plone.app.upgrade.bbb import ITinyMCE
sm.unregisterUtility(provided=ITinyMCE)
import transaction; transaction.commit()

If it's successful, you should be able to look at the Components tab of the Plone site in the ZMI, and see no reference to ITinyMCE there.

dieter · April 8, 2023, 9:52pm

Andrew Calcutt via Plone Community wrote at 2023-4-8 20:01 +0000:

davisagli:

portal_tinymce

I had deleted portal_tinymce in the py2 to py3 migration steps in the fixes for https://community.plone.org/t/upgrade-to-5-2-failing-with-iatcttool-has-no-attribute-iro

How would I clean up its registration as a utility in 5.2?

You could try wildcard.fixpersistentregistries (or similarly spelled).

acalcutt · April 11, 2023, 1:26pm

@davisagli

I ran these steps and it helped with 3/4 of my sites. 3 of them upgraded just fine now. The other one still gives me a plone.app.upgrade.bbb.ITinyMCE error. (of course this is my most used and longest running site)

I also tried the cleaning this up using wildcard.fixpersistentregistries , but I had to go back to my 5.1.6 instance for that since 5.2 (python2 or python3) didn't seem to find it. I did delete an instance of plone.app.upgrade.bbb.ITinyMCE there and run the debug install steps, but the issue still persisted.

I then decided to try making a new site in 5.2 and copying over the data I care about, but I kept running into another error

2023-04-11 13:07:15,425 ERROR [plone.app.upgrade:306][waitress-1] Upgrade aborted. Error:
Traceback (most recent call last):
File "/opt/Plone_6/eggs/Products.CMFPlone-6.0.3-py3.8.egg/Products/CMFPlone/MigrationTool.py", line 299, in upgrade
step['step'].doStep(setup)
File "/opt/Plone_6/eggs/Products.GenericSetup-3.0.1-py3.8.egg/Products/GenericSetup/upgrade.py", line 193, in doStep
self.handler(tool)
File "/opt/Plone_6/eggs/plone.app.upgrade-3.0.3-py3.8.egg/plone/app/upgrade/v60/alphas.py", line 175, in fix_unicode_properties
portal.ZopeFindAndApply(portal, search_sub=1, apply_func=fix_properties)
File "/opt/Plone_6/eggs/Zope-5.8.1-py3.8.egg/OFS/FindSupport.py", line 171, in ZopeFindAndApply
self.ZopeFindAndApply(ob, obj_ids, obj_metatypes,
File "/opt/Plone_6/eggs/Zope-5.8.1-py3.8.egg/OFS/FindSupport.py", line 165, in ZopeFindAndApply
apply_func(ob, (apply_path + '/' + p))
File "/opt/Plone_6/eggs/Zope-5.8.1-py3.8.egg/ZPublisher/utils.py", line 165, in fix_properties
current = obj.getProperty(prop_id)
File "/opt/Plone_6/eggs/Zope-5.8.1-py3.8.egg/OFS/PropertyManager.py", line 157, in getProperty
return getattr(self, id)
AttributeError: 'RequestContainer' object has no attribute 'layout'

I actually figured this one out in a very hacky way. I went and edited the (File "/opt/Plone_6/eggs/Zope-5.8.1-py3.8.egg/OFS/PropertyManager.py", line 157) it was complain about and added in two print statements into the function so I could see where it was erroring.
.

With that change in place I reran the upgrade and right before the error it would report the the object and id of the thing it was working on, like

I then realized if I tried to view the properties of that "autoit-scripts" folder in zope on my 5.2 instance, I got a similar error about the layout property. Since I could still get to all the contents in 5.2, I decided to rename the folder and make a new folder with the old name. the I copied all the contents on the old folder into the new one , and deleted the old folder. once removing the old folder with the bad layout property, the upgraded succeeded... so I have pretty much everything I care about moved to Plone 6 now....

I think at this point I could be happy with this new copied site and call it a day. One thing I am missing is the ability to always show the left navigation, not just after a top link is clicked.... I have no idea how I did that before.

acalcutt · April 11, 2023, 3:31pm

I actually managed to get past the original " Can’t pickle class ‘plone.app.upgrade.bbb.ITinyMCE' error in the last site

I was following steps from this article linked in another thread, Removing a persistent local utility part II - Four Digits

I went through the steps there in "bin/instance debug" and found there was still a ‘plone.app.upgrade.bbb.ITinyMCE' subscriber.

In the end I ran the following in the debug console, which seemed to finally fix this error

sm = app.TechIdiots.getSiteManager()
adapters = sm.utilities._adapters
for x in adapters[0].keys():
    if x.__module__.find("plone.app.upgrade") != -1:
        print("deleting %s" % x)
        del adapters[0][x]


sm.utilities._adapters = adapters

subscribers = sm.utilities._subscribers
for x in subscribers[0].keys():
    if x.__module__.find("plone.app.upgrade") != -1:
        print("deleting %s" % x)
        del subscribers[0][x]


sm.utilities._subscribers = subscribers

provided = sm.utilities._provided
for x in provided.keys():
    if x.__module__.find("plone.app.upgrade") != -1:
        print("deleting %s" % x)
        del provided[x]


sm.utilities._provided = provided

from transaction import commit
commit()
app._p_jar.sync()

When running that, i got back 'deleting plone.app.upgrade.bbb.ITinyMCE' on the subscribers section.

With this, plus the folder layout fixes I already found, I was finally able to get the site to upgrade without copying all the files to a new site.

davisagli · April 11, 2023, 3:53pm

@acalcutt What a pain. Congrats on getting the site working!

acalcutt · April 11, 2023, 4:04pm

The sad thing is after all that, I'm still not sure I like what the plone 6 upgrade did to my layout... I still have to research that to see if I can get my side menu back before I decided to use it...

At least I know my 5.2 instance can be upgraded now.

davisagli · April 11, 2023, 4:21pm

Usually it's a matter of setting the Navigation portlet to have a "Top level" of 0. But, I'm not sure why that would have been changed during the migration.

acalcutt · April 11, 2023, 5:07pm

I checked the navigation portlet and it is still at a start level of zero. It honestly doesn't seem like the left navigation is working at all on any of the sites I migrated. Even if I click a subfolder it doesn't show.

This might be a new discussion I guess

davisagli · April 11, 2023, 5:55pm

Is there any error in the instance log? I tried the navigation portlet on Welcome to Plone — English and it seems to be working there.

acalcutt · April 11, 2023, 6:27pm

I'm guessing it is related to this error in the console that i see when the page loads.

 'navigation'): '<' not supported between instances of 'str' and 'ComputedAttribute'
Traceback (most recent call last):
  File "/opt/Plone_6/eggs/plone.portlets-2.3.3-py3.8.egg/plone/portlets/manager.py", line 119, in _lazyLoadPortlets
    isAvailable = renderer.available
  File "/opt/Plone_6/eggs/plone.app.portlets-5.0.2-py3.8.egg/plone/app/portlets/portlets/navigation.py", line 228, in available
    rootpath = self.getNavRootPath()
  File "/opt/Plone_6/eggs/plone.memoize-3.0.0-py3.8.egg/plone/memoize/instance.py", line 53, in memogetter
    val = func(*args, **kwargs)
  File "/opt/Plone_6/eggs/plone.app.portlets-5.0.2-py3.8.egg/plone/app/portlets/portlets/navigation.py", line 313, in getNavRootPath
    return getRootPath(
  File "/opt/Plone_6/eggs/plone.app.portlets-5.0.2-py3.8.egg/plone/app/portlets/portlets/navigation.py", line 515, in getRootPath
    root = uuidToObject(root)
  File "/opt/Plone_6/eggs/plone.app.uuid-2.2.2-py3.8.egg/plone/app/uuid/utils.py", line 85, in uuidToObject
    path = uuidToPhysicalPath(uuid)
  File "/opt/Plone_6/eggs/plone.app.uuid-2.2.2-py3.8.egg/plone/app/uuid/utils.py", line 47, in uuidToPhysicalPath
    rid = index._index.get(uuid)
TypeError: '<' not supported between instances of 'str' and 'ComputedAttribute'

That error leads me to missing root_uuid of navigation-portlet throws error in DB migrated to py3 · Issue #2876 · plone/Products.CMFPlone · GitHub . I was going to try the collective.migrationhelpers mentioned there , but it didn't seem to find it when i added it to my buildout.

BTW, I made a new plone site and the menu does work in that one. It's just the ones I upgraded that the navigation is not working.

davisagli · April 11, 2023, 6:45pm

@acalcutt As a workaround can you add a new Navigation portlet, and then delete the old one?

acalcutt · April 11, 2023, 7:16pm

Nice, I renamed the old portlet, added a new one, then deleted the old one and it worked!