Upgrade from 3.3.3 - can't get rid of iw.fss.modifier.Modifier

fulv · December 22, 2015, 6:30am

I'm trying to upgrade a Plone 3.3.3 site - initially to 4.3.7, and then evaluate further upgrade to 5.latest.
I have uninstalled all add-ons except PloneFormGen and plone.app.blob, and removed all content created with any of the uninstalled add-ons. I also used fixpersistentutilities to get rid of P4A crap.

When I run the upgrade in Plone 4.3.7, it fails. I've pasted the tail of the log.

I understand that the first four Could not import class lines are a problem (though I don't know how to deal with those). But I'm also concerned with the

TypeError: ('object.__new__(Modifier) is not safe, use persistent.Persistent.__new__()', <function _reconstructor at 0x10da79cf8>, (<class 'iw.fss.modifier.Modifier'>, <type 'object'>, None))

I would like to delete the offending object, so I put a breakpoint in Python-2.7/lib/python2.7/copy_reg.py", line 48, in _reconstructor, but I can't figure out what object is causing this.

To top things off, when I try to delete some invalid steps in portal_setup/manage_stepRegistry, I get a NotFound - The object at http://localhost:8380/cms/portal_setup/manage_stepRegistry/manage_deleteExportSteps has an empty or missing docstring. Objects must have a docstring to be published. I'm guessing that this is because some object along the chain is None, but that's as far as I get.

Questions:

How do I identify and clean up the object causing the TypeError to be raised?
How do I deal with the missing Contentish, WriteLockInterface, DynamicType and even Modifier classes?
How do I work around the broken deleteExportSteps?
Am I just wasting my time and should I use transmogrifier instead?

If anyone has any suggestions at all, I would be grateful.
Thanks!

============================================================================

2015-12-21 19:22:51 WARNING OFS.Uninstalled Could not import class 'Contentish' from module 'Products.CMFCore.interfaces.Contentish' 2015-12-21 19:22:51 WARNING OFS.Uninstalled Could not import class 'WriteLockInterface' from module 'webdav.WriteLockInterface' 2015-12-21 19:22:51 WARNING OFS.Uninstalled Could not import class 'DynamicType' from module 'Products.CMFCore.interfaces.Dynamic' 2015-12-21 19:22:57 WARNING OFS.Uninstalled Could not import class 'Modifier' from module 'iw.fss.modifier' 2015-12-21 19:22:57 ERROR ZODB.Connection Couldn't load state for 0x5005 Traceback (most recent call last): File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/Connection.py", line 860, in setstate self._setstate(obj) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/Connection.py", line 914, in _setstate self._reader.setGhostState(obj, p) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/serialize.py", line 612, in setGhostState state = self.getState(pickle) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/serialize.py", line 605, in getState return unpickler.load() File "/Users/fulvio/python/plone/Plone-4.3.7/Python-2.7/lib/python2.7/copy_reg.py", line 48, in _reconstructor obj = object.__new__(cls) TypeError: ('object.__new__(Modifier) is not safe, use persistent.Persistent.__new__()', <function _reconstructor at 0x10da79cf8>, (<class 'iw.fss.modifier.Modifier'>, <type 'object'>, None)) 2015-12-21 19:22:57 ERROR plone.app.upgrade Upgrade aborted. Error: Traceback (most recent call last): File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/Products.CMFPlone-4.3.7-py2.7.egg/Products/CMFPlone/MigrationTool.py", line 259, in upgrade step['step'].doStep(setup) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/Products.GenericSetup-1.7.7-py2.7.egg/Products/GenericSetup/upgrade.py", line 160, in doStep self.handler(tool) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/plone.app.upgrade-1.3.18-py2.7.egg/plone/app/upgrade/v40/alphas.py", line 480, in migrateFolders MigrationView(portal, None)() File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/plone.app.folder-1.1.0-py2.7.egg/plone/app/folder/migration.py", line 81, in __call__ self.postprocess(obj) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/plone.app.upgrade-1.3.18-py2.7.egg/plone/app/upgrade/v40/alphas.py", line 467, in postprocess meta_type = getattr(aq_base(obj), 'meta_type', None) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/Connection.py", line 860, in setstate self._setstate(obj) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/Connection.py", line 914, in _setstate self._reader.setGhostState(obj, p) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/serialize.py", line 612, in setGhostState state = self.getState(pickle) File "/Users/fulvio/python/plone/Plone-4.3.7/buildout-cache/eggs/ZODB3-3.10.5-py2.7-macosx-10.11-intel.egg/ZODB/serialize.py", line 605, in getState return unpickler.load() File "/Users/fulvio/python/plone/Plone-4.3.7/Python-2.7/lib/python2.7/copy_reg.py", line 48, in _reconstructor obj = object.__new__(cls) TypeError: ('object.__new__(Modifier) is not safe, use persistent.Persistent.__new__()', <function _reconstructor at 0x10da79cf8>, (<class 'iw.fss.modifier.Modifier'>, <type 'object'>, None)) 2015-12-21 19:22:57 INFO plone.app.upgrade Starting upgrade of core addons. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile Products.CMFEditions:CMFEditions is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile Products.CMFPlacefulWorkflow:CMFPlacefulWorkflow is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile Products.TinyMCE:TinyMCE is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.dexterity:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.discussion:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.iterate:plone.app.iterate is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.jquery:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.jquerytools:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.querystring:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plone.app.theming:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 WARNING Products.GenericSetup.tool Version of profile plonetheme.sunburst:default is unknown, refusing to upgrade. 2015-12-21 19:22:57 INFO plone.app.upgrade Done upgrading core addons. 2015-12-21 19:22:57 INFO plone.app.upgrade End of upgrade path, migration has finished 2015-12-21 19:22:57 ERROR plone.app.upgrade The upgrade path did NOT reach current version 2015-12-21 19:22:57 ERROR plone.app.upgrade Migration has failed

polyester · December 22, 2015, 7:06am

had similar experiences with a non-standard storage, that couldn't find the stored objects before uninstalling the product and thus left bits and pieces lying around.

I would (as hunch) first look into iw.fss; if you uninstall that you can't get rid of objects that used it (since the FileSystemStorage is not there anymore) and stuff breaks. It also looks like it did not uninstall cleanly.

fulv · December 22, 2015, 11:04pm

Hi Paul,

Thanks for your reply, you pointed me in the right direction. Turns out, iw.fss was not one of the add-ons I had uninstalled personally in preparation for the upgrade. Apparently, the client had removed its code without uninstalling it, or botched the uninstallation. What threw me off track was the fact that iw.fss was not listed in portal_quickinstaller (neither as installed, nor as installable), but someone put the egg in the products directory, so I thought it was present. I thought it was the old-style format of product, but no, it turned out to be an egg, so it was getting ignored.

Ultimately, I resolved this by

Adding iw.fss to the eggs and zcml in buildout (and re-running buildout),
After this, iw.fss showed up in portal_quickinstaller as installed. So I could uninstall it from there.
Retrying the 4.3.7 upgrade. This was enough to make it succeed.

Now I'm getting a ImportError: No module named SQLAlchemyDA.da when I try to do a zexp export of the site, but that's tomorrow's battle.

Cheers,
Fulvio