Collective.xdv breaks ugprade from 4.0.9 to 4.3

Plone Support
,
1

Hi

I am trying to upgrade from {Plone 4.0.9 running bare metal and upgraded when I really have to} to {Plone 4.3 in Docker containers orchestrated from GitLab and upgraded frequently}. I do not mind sharing my Dockerfile, docker-compose.yml and .gitlab-ci.yml when I am done, if anybody is interested.

I am hitting a roadblock that my limited knowledge of Plone does not help me solve. In a nutshell, the upgrade to 4.3 fails with PicklingError: Can't pickle <class 'collective.xdv.interfaces.IXDVLayer'>: import of module collective.xdv.interfaces failed

The least unreasonable workflow I have tried so far has been:

  1. Patch QuickInstallerTool.py shipped with 4.0.9 to avoid cascading removal of utilities products:
@@ -608,6 +608,8 @@ class QuickInstallerTool(UniqueObject, O
                           REQUEST=None):
         """Removes a list of products
         """
+        if "utilities" in cascade:
+            cascade.remove("utilities")
         for pid in products:
             prod=getattr(self,pid)
             prod.uninstall(cascade=cascade, reinstall=reinstall)

This steps prevents the pesky ComponentLookupError: (<InterfaceClass plone.registry.interfaces.IRegistry>, '') issue.

  1. Restore the production (4.0.9) database to my (bare metal, 4.0.9) development environment
  2. From the ZMI, QuickInstaller, uninstall 'XDV theme support'
  3. Using /@@fix-persistent-utilities, remove InterfaceClass collective.xdv.interfaces.IXDVLayer (two occurences)
  4. bin/plonectl stop
  5. Remove collective.xdv from buildout.cfg, run bin/buildout -Nv
  6. bin/plonectl start to have a look around and confirm that the collective.xdv stuff is gone from /prefs_install_products_form and /@@fix-persistent-utilities.
  7. bin/plonectl stop, back up the database
  8. Restore the database in the containerized 4.3.
  9. Hit the "upgrade" button and stare at the containers logs scrolling until the dreadfull
2017-12-24 04:04:56 INFO GenericSetup.rolemap Role / permission map imported.
2017-12-24 04:04:56 INFO GenericSetup.catalog Catalog imported.
2017-12-24 04:04:56 INFO GenericSetup.archetypetool Archetype tool imported.
2017-12-24 04:04:56 ERROR GenericSetup Step pleonformgen has an invalid import handler
2017-12-24 04:04:56 ERROR GenericSetup Step kupu-setup has an invalid import handler
2017-12-24 04:04:56 INFO plone.app.upgrade Reindex Title index with I18N Case Normalizer
2017-12-24 04:04:59 ERROR plone.app.upgrade Upgrade aborted. Error:
Traceback (most recent call last):
  File "/plone/buildout-cache/eggs/Products.CMFPlone-4.3.14-py2.7.egg/Products/CMFPlone/MigrationTool.py", line 259, in upgrade
    step['step'].doStep(setup)
  File "/plone/buildout-cache/eggs/Products.GenericSetup-1.8.6-py2.7.egg/Products/GenericSetup/upgrade.py", line 166, in doStep
    self.handler(tool)
  File "/plone/buildout-cache/eggs/plone.app.upgrade-1.4.1-py2.7.egg/plone/app/upgrade/v43/betas.py", line 11, in to43rc1
    upgradeToI18NCaseNormalizer(context)
  File "/plone/buildout-cache/eggs/plone.app.upgrade-1.4.1-py2.7.egg/plone/app/upgrade/v43/alphas.py", line 67, in upgradeToI18NCaseNormalizer
    aq_get(context, 'REQUEST', None))
  File "/plone/buildout-cache/eggs/Products.ZCatalog-2.13.27-py2.7.egg/Products/ZCatalog/ZCatalog.py", line 436, in reindexIndex
    update_metadata=0, pghandler=pghandler)
  File "/plone/buildout-cache/eggs/Products.CMFPlone-4.3.14-py2.7.egg/Products/CMFPlone/CatalogTool.py", line 349, in catalog_object
    update_metadata, pghandler=pghandler)
  File "/plone/buildout-cache/eggs/Products.ZCatalog-2.13.27-py2.7.egg/Products/ZCatalog/ZCatalog.py", line 500, in catalog_object
    transaction.savepoint(optimistic=True)
  File "/plone/buildout-cache/eggs/transaction-1.1.1-py2.7.egg/transaction/_manager.py", line 101, in savepoint
    return self.get().savepoint(optimistic)
  File "/plone/buildout-cache/eggs/transaction-1.1.1-py2.7.egg/transaction/_transaction.py", line 260, in savepoint
    self._saveAndRaiseCommitishError() # reraises!
  File "/plone/buildout-cache/eggs/transaction-1.1.1-py2.7.egg/transaction/_transaction.py", line 257, in savepoint
    savepoint = Savepoint(self, optimistic, *self._resources)
  File "/plone/buildout-cache/eggs/transaction-1.1.1-py2.7.egg/transaction/_transaction.py", line 690, in __init__
    savepoint = savepoint()
  File "/plone/buildout-cache/eggs/ZODB3-3.10.7-py2.7-linux-x86_64.egg/ZODB/Connection.py", line 1125, in savepoint
    self._commit(None)
  File "/plone/buildout-cache/eggs/ZODB3-3.10.7-py2.7-linux-x86_64.egg/ZODB/Connection.py", line 623, in _commit
    self._store_objects(ObjectWriter(obj), transaction)
  File "/plone/buildout-cache/eggs/ZODB3-3.10.7-py2.7-linux-x86_64.egg/ZODB/Connection.py", line 658, in _store_objects
    p = writer.serialize(obj)  # This calls __getstate__ of obj
  File "/plone/buildout-cache/eggs/ZODB3-3.10.7-py2.7-linux-x86_64.egg/ZODB/serialize.py", line 422, in serialize
    return self._dump(meta, obj.__getstate__())
  File "/plone/buildout-cache/eggs/ZODB3-3.10.7-py2.7-linux-x86_64.egg/ZODB/serialize.py", line 431, in _dump
    self._p.dump(state)
PicklingError: Can't pickle <class 'collective.xdv.interfaces.IXDVLayer'>: import of module collective.xdv.interfaces failed
2017-12-24 04:04:59 INFO plone.app.upgrade End of upgrade path, main migration has finished.
2017-12-24 04:04:59 ERROR plone.app.upgrade The upgrade path did NOT reach current version.
2017-12-24 04:04:59 ERROR plone.app.upgrade Migration has failed

Anybody with ideas on how to go past this?

2

What I think might help is the removal of the browser layer (IXDVLayer) using an uninstallation step.
At first glance, it looks like fixpersistent utilities should do this but.......
This can be achieved by registering and using an uninstall step.
More information here: http://blog.keul.it/2013/05/how-to-make-your-plone-add-on-products.html
(look for the section that discusses "Uninstallation").

1 Like
3

Thanks for the link @pigeonflight, it is indeed a good read.

I tweaked collective/xdv/configure.zcml to add

    <gs:registerProfile
        name="uninstall"
        title="XDV theme support"
        description="Uninstall XDV"
        directory="profiles/uninstall"
        provides="Products.GenericSetup.interfaces.EXTENSION"
        />

I created a collective/xdv/Extensions folder and a file collective/xdv/Extensions/install.py:

from Products.CMFCore.utils import getToolByName
import logging
logger = logging.getLogger('collective.xdv')

def uninstall(portal):
    setup_tool = getToolByName(portal, 'portal_setup')
    setup_tool.runAllImportStepsFromProfile('profile-collective.xdv:uninstall')
    logger.warning("Guillaume - uninstall of profile-collective.xdv.interfaces.IXDVLayer done")
    return "Ran all uninstall steps."

However, Plone seems to be gloriously ignoring my install.py. Nothing in the logs when I uninstall the product, but I do see the expected logs when I install it:

2017-12-24T17:42:54 INFO GenericSetup.browserlayer Browser layers imported

Does install.py have to be declared or registered anywhere?

4

Try using portal_setup to run the uninstall profile.
{your-site}/portal_setup/manage > import

5

I can see the uninstall profile and run it from {my-site}/portal_setup/manage > import. None of the "available import steps" rings a bell, so I tried to run all of them. They do not produce the logs I would expect, and do not seem to be calling my install.py uninstall.

6

Maybe I'm missing something.
If that isn't working for you, another approach is to use a genericsetup profile, similar to what is done here: https://github.com/collective/collective.pdfpeek/blob/master/collective/pdfpeek/profiles/uninstall/browserlayer.xml

The browserlayer.xml can be used to remove specific layers.