Small project migration with customized barceloneta

Over the holidays I migrated the very small website from Plone 4.3 with Archetypes and LinguaPlone to Plone 5.2 with Dexterity on Python 3.8.

The project had 2 interesting aspects to it:

The Migration

The migration was finished rather quick in less than 2 days. The site only had a small number of addons that mostly had become obsolete for the project: Products.ImageEditor, webcouturier.dropdownmenu, collective.flowplayer plus custom policy and theme-packages. For small to medium projects I prefer to have only one package (in this case that holds everything)
Since the additional languages were never published (the usual story: the client never had enough time to deal with the additional work) I made it one-language only now. I will show the code at the end of this post. Most of it was taken from with only small additions.

The process is the usual:

  • Remove obsolete Addons, cleanup and remove LinguaPlone in Plone 4.3
  • Upgrade to Plone 5.2
  • Migrate to Dexterity
  • Migrate to Python 3

The migration to Python 3 was nothing more than running

./bin/zodbupdate --convert-py3 --file=var/filestorage/Data.fs --encoding=utf8

The Theme

The old theme was done very old school by overwriting the sunburst theme. For the new theme I decided to try the same approach with barceloneta. The whole theme is only 69 css-rules that overwrite barceloneta, mostly the navigation and the portlets. Two problems coule not be solves with css only:

  • The grid: column-one (that holds the left portlets) needs a width of 4 instead of thge default 3.
  • The main-navigation needs to be in the header-container.

So I made a ttw-copy of the barceloneta-theme and modified two things:

  • In index.html I moved mainnavigation-wrapper up into the header with a width of col-md-8
  • In rules.xml I changed the conditionally modified css-class of portal-column-one to col-xs-12 col-sm-12 col-md-8 col-md-push-4 (indead of col-md-8 col-md-push-4).

The portal-header got its class col-md-4 by overriding with jbot.

I like the approach to keep modifications as small as possible because that makes the next upgrade easier. The whole package has almost no code, some useful registry settings and a bit of css.
The copy of the theme in the DB is bothering me a little but since the diff is so small (and I stored it diff) it will be easy to upgrade to future version of barceloneta. I would have preferred to only overwrite rules.xml and index.html in my package but that is not possible.

It was done quick but not very dirty and deployed on December 30, 2019.


Here is the migration-code run on Plone 4.3:

# -*- coding: utf-8 -*-
from plone import api
from plone.portlets.interfaces import IPortletAssignmentMapping
from plone.portlets.interfaces import IPortletManager
from zope.component import getMultiAdapter
from zope.component import getUtility
from zExceptions import BadRequest

import logging
import transaction

log = logging.getLogger(__name__)

def prepare_plone5_upgrade(setup):
    portal = api.portal.get()
    catalog = api.portal.get_tool('portal_catalog')
    qi = api.portal.get_tool('portal_quickinstaller')
    portal_skins = api.portal.get_tool('portal_skins')
    portal_properties = api.portal.get_tool('portal_properties')

    to_delete = [
    for path in to_delete:
            obj = api.content.get(path=path)
            if obj is not None:
                api.content.delete(obj, check_linkintegrity=False)
  'Deleted %s' % path)

    # Remove webcouturier.dropdownmenu
    if qi.isProductInstalled('webcouturier.dropdownmenu'):

    # remove Products.ImageEditor'removing Products.ImageEditor')
    if qi.isProductInstalled('ImageEditor'):

    if qi.isProductInstalled(''):

    # collective.flowplayer'removing collective.flowplayer')
    if qi.isProductInstalled('collective.flowplayer'):

    # collective.js.fancybox
    if qi.isProductInstalled('collective.js.fancybox'):

    # collective.revisionmanager
    if qi.isProductInstalled('collective.revisionmanager'):
        qi.uninstallProducts(['collective.revisionmanager'])'rebuilding catalog')

    manager = getUtility(
        IPortletManager, name='plone.leftcolumn', context=portal)
    mapping = getMultiAdapter((portal, manager), IPortletAssignmentMapping)
    for mapping_id in mapping:
            del mapping[mapping_id]
        except KeyError:
            pass'Set languages')
    for brain in catalog.unrestrictedSearchResults():
        obj = brain.getObject()
        if obj.Language() == 'en':
  'Deleteting english content %s' % obj.absolute_url())
            api.content.delete(obj, check_linkintegrity=False)
        elif obj.Language() != 'de':
        if obj.portal_type == 'WildcardVideo':
  'Deleteting %s' % obj.absolute_url())
            api.content.delete(obj, check_linkintegrity=False)
    transaction.commit()'remove LP')
    if qi.isProductInstalled('LinguaPlone'):
    except BadRequest:

    transaction.commit()'run uninstall self')
    # run our uninstall-profile that also removes LP
        'profile-plonetheme.stiftung:uninstall', purge_old=False)'unregister_broken_persistent_components')

def remove_vocabularies(setup):
    from plone.i18n.locales.interfaces import IContentLanguageAvailability
    from plone.i18n.locales.interfaces import IMetadataLanguageAvailability
    portal = api.portal.get()
    sm = portal.getSiteManager()

    if IContentLanguageAvailability in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IContentLanguageAvailability]'Unregistering subscriber for IContentLanguageAvailability')
    if IMetadataLanguageAvailability in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IMetadataLanguageAvailability]'Unregistering subscriber for IMetadataLanguageAvailability')

    if IMetadataLanguageAvailability in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IMetadataLanguageAvailability]'Unregistering adapter for IMetadataLanguageAvailability')
    if IContentLanguageAvailability in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IContentLanguageAvailability]'Unregistering adapter for IContentLanguageAvailability')

    sm.utilities._p_changed = True

def unregister_broken_persistent_components(portal):
    sm = portal.getSiteManager()

    for item in sm._utility_registrations.items():
        if hasattr(item[1][0], '__Broken_state__'):
            # unregisterUtility(component, provided, name)
            # See: five.localsitemanager.registry.PersistentComponents.unregisterUtility  # noqa: E501
  "Unregistering component {0}".format(item))
            sm.unregisterUtility(item[1][0], item[0][0], item[0][1])

def remove_overrides(context=None):'removing portal_skins overrides')
    portal_skins = api.portal.get_tool('portal_skins')
    custom = portal_skins['custom']
    for name in custom.keys():
        custom.manage_delObjects([name])'Removed skin item {}'.format(name))'removing portal_view_customizations')
    view_customizations = api.portal.get_tool('portal_view_customizations')
    for name in view_customizations.keys():
        view_customizations.manage_delObjects([name])'Removed portal_view_customizations item {}'.format(name))

def release_all_webdav_locks(context=None):
    from Products.CMFPlone.utils import base_hasattr
    portal = api.portal.get()

    def unlock(obj, path):
        if base_hasattr(obj, 'wl_isLocked') and obj.wl_isLocked():
  'Unlocked {}'.format(path))

    portal.ZopeFindAndApply(portal, search_sub=True, apply_func=unlock)

def remove_all_revisions(context=None):
    """Remove all revisions.
    After packing the DB this could significantly shrink its size.
    hs = api.portal.get_tool('portal_historiesstorage')
    zvcr = hs.zvc_repo
    storage = hs._shadowStorage

def disable_theme(context=None):
    """Disable a custom diazo theme and enable sunburst.
    Useful for cleaning up a site in Plone 4
    THEME_NAME = 'plonetheme.stiftung'
    from import applyTheme
    portal_skins = api.portal.get_tool('portal_skins')
    qi = api.portal.get_tool('portal_quickinstaller')
    if qi.isProductInstalled(THEME_NAME):'Uninstalling {}'.format(THEME_NAME))
        qi.uninstallProducts([THEME_NAME])'Disabling all diazo themes')
    # applyTheme(None)'Enabled Sunburst Theme')
    portal_skins.default_skin = 'Sunburst Theme'
    if THEME_NAME in portal_skins.getSkinSelections():
        portal_skins.manage_skinLayers([THEME_NAME], del_skin=True)

Here is the migration-code run on Plone 5.2 (on Python 2):

def prep_dx_migration(context=None):
    portal = api.portal.get()
    request = getRequest()
    installer = api.content.get_view('installer', portal, request)
    if not installer.is_product_installed(''):

def migrate_to_dexterity(setup):
    portal = api.portal.get()
    request = getRequest()
    pac_migration = api.content.get_view('migrate_from_atct', portal, request)

def cleanup_after_migration_to_p5(setup):
    portal = api.portal.get()
    request = getRequest()
    installer = api.content.get_view('installer', portal, request)
    addons = [
    for addon in addons:
        if installer.is_product_installed(addon):
    portal._delObject('archetype_tool', suppress_events=True)
    portal._delObject('reference_catalog', suppress_events=True)
    portal._delObject('uid_catalog', suppress_events=True)

    portal_properties = portal.portal_properties

    if not installer.is_product_installed('plonetheme.barceloneta'):

    broken_import_steps = [
    registry = setup.getImportStepRegistry()
    for broken_import_step in broken_import_steps:
        if broken_import_step in registry.listSteps():

def cleanup_after_at_removal(setup):
    portal = api.portal.get()

def unregister_broken_persistent_components(portal):
    sm = portal.getSiteManager()

    for item in sm._utility_registrations.items():
        if hasattr(item[1][0], '__Broken_state__'):
            # unregisterUtility(component, provided, name)
            # See: five.localsitemanager.registry.PersistentComponents.unregisterUtility  # noqa: E501
  "Unregistering component {0}".format(item))
            sm.unregisterUtility(item[1][0], item[0][0], item[0][1])

    from Products.ATContentTypes.interfaces.interfaces import IATCTTool
    from Products.CMFCore.interfaces import IMetadataTool
    portal = api.portal.get()
    sm = portal.getSiteManager()

    if IMetadataTool in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IMetadataTool]'Unregistering subscriber for IMetadataTool')
    if IATCTTool in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IATCTTool]'Unregistering subscriber for IATCTTool')

    if IATCTTool in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IATCTTool]'Unregistering adapter for IATCTTool')
    if IMetadataTool in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IMetadataTool]'Unregistering adapter for IMetadataTool')

    if IATCTTool in sm.utilities._provided:
        del sm.utilities._provided[IATCTTool]'Unregistering _provided for IATCTTool')
    if IMetadataTool in sm.utilities._provided:
        del sm.utilities._provided[IMetadataTool]'Unregistering _provided for IMetadataTool')

    sm.utilities._p_changed = True

def restructure_content(context=None):
    portal = api.portal.get()
    request = getRequest()

    de = portal['de']
    for key in de:
        api.content.move(source=de[key], target=portal)

    manage_viewlets = api.content.get_view('manage-viewlets', portal, request)
    # hide search viewlet
    manage_viewlets.hide('plone.portalheader', 'plone.searchbox')
    manage_viewlets.hide('plone.mainnavigation', 'plone.path_bar')

    # Move leadimage above breadcrumbs
            'plone.mainnavigation', 'plone.path_bar', 'plone.global_sections'
    except ValueError:


Thanks to a change in z3c.jbot 1.1.0 I was today able to remove the copy of the theme from the database. Now I only override the files from barceloneta as plonetheme.barceloneta.theme.index.html and plonetheme.barceloneta.theme.rules.xml.


I am trying to figure out how to use the code that @pbauer has written and generously shared here and on GitHub. I think that I am missing a basic concept. Hopefully someone can help me with this.

In both of the code examples that Philip has provided in his post above, I see where he defines several Python functions but I do not see where he calls them and in what order I should call them. While I have not dug into the details of his code, I am able to somewhat understand what it does by reading through it. I am missing how I am supposed to use his code to make my Plone site run better. I tried copying and pasting his second code block into and running it my Python2.7 instance of Plone 5.2.5 with

cd /opt/plone5.2.5_python2.7/zinstance
cat >
sudo -u plone_daemon bin/instance run

and the code appears to run yet I do not see any output and I don't see any evidence that it change my site. I was hopeful that running it would somehow fix my site generating the AttributeError: type object 'IKSSRegistry' has no attribute '__iro__' error message when trying to run @@plone-upgrade.

Can someone give me an example of how to use Philip's code?


please post the code.

Thanks @yurj. I was trying to run the code that @pbauer posted above. I would really like to run @pbauer's remove_archetypes function.

# -*- coding: UTF-8 -*-
from plone import api

import logging

log = logging.getLogger(__name__)

def remove_archetypes(context=None):
    portal = api.portal.get()

    # remove obsolete AT tools
    tools = [
    for tool in tools:
        if tool not in portal.keys():
  'Tool {} not found'.format(tool))
  'Deleted {}'.format(tool))
        except Exception as e:
  'Problem removing {}: {}'.format(tool, e))
      'Fallback to remove without permission_checks')
      'Deleted {}'.format(tool))
            except Exception as e:
      'Another problem removing {}: {}'.format(tool, e))

def fix_conversations(context=None):
    """Conversations from may still have the old
    Archetypes content as __parent__ instead of the new migrated DX-content.
    # TODO: Fix in
    from Acquisition import aq_base
    portal = api.portal.get()

    def fix_conversation_parent(obj, path):
        annotations = getattr(aq_base(obj), '__annotations__', None)
        if not annotations:
        if '' not in annotations.keys():
        conversation = annotations['']
        if 'broken' in conversation.__parent__.__repr__() or obj != conversation.__parent__:
            conversation.__parent__ = obj
  'Fix conversation for {}'.format(obj.absolute_url()))
    portal.ZopeFindAndApply(portal, search_sub=True, apply_func=fix_conversation_parent)

I found the 16.6. Mr. Devloper section in the Mastering Plone 5 Development course. I tried to install collective.migrationhelpers by updating my site's buildout.cfg and running sudo -u plone_buildout bin/buildout. Buildout ran fine but when I run sudo -u plone_daemon ./bin/instance fg I don't see remove_archetypes listed in the ZMI. I think I am missing a step.

diff --git a/buildout.cfg b/buildout.cfg
index e89fbe4..d17e652 100644
--- a/buildout.cfg
+++ b/buildout.cfg
@@ -26,8 +26,20 @@
 # buildout.sanitycheck makes sure you're not running buildout
 # as root.
-# extensions =
-#     buildout.sanitycheck
+extensions =
+    buildout.sanitycheck
+    mr.developer
+# Tell mr.developer to ask before updating a checkout.
+always-checkout = true
+show-picked-versions = true
+sources = sources
+# The directory this buildout is in. Modified when using vagrant.
+buildout_dir = ${buildout:directory}
+# We want to checkouts these eggs directly from github
+auto-checkout =
+    collective.migrationhelpers
 # Plone Component Versions
@@ -69,6 +81,9 @@ need-sudo = yes
 eggs =
+    Products.ResourceRegistries
+    Plone [archetypes]
+    collective.migrationhelpers
@@ -189,3 +204,9 @@ collective.recipe.backup = 4.1.0
 # only for Windows
 nt-svcutils = 2.13.0
+collective.migrationhelpers = git

While I have tried reading through remove_archetypes and doing the same things by hand in the ZMI, I think I am not sure if I am doing the right things and would like to run the code and see if it works better than running it by hand.

I would appreciate any help you can give me with running remove_archetypes.


you're on the right path! Those are functions, so you've to just run them. In the code above, at the end, just add:


or the function you need.

@yurj Thanks!

I tried adding remove_archetypes() to the end of and it generated a plone.api.exc.CannotGetPortalError: Unable to get the portal object.

cd /opt/plone5.2.5_python2.7.clean/zinstance
cp /opt/collective.migrationhelpers/src/collective/migrationhelpers/ .
# Added remove_archetypes() to the end of
# Ran with
root@plone2:/opt/plone5.2.5_python2.7.clean/zinstance# sudo -u plone_daemon bin/instance run
Traceback (most recent call last):
  File "/opt/plone5.2.5_python2.7.clean/zinstance/parts/instance/bin/interpreter", line 295, in <module>
  File "<string>", line 1, in <module>
  File "", line 65, in <module>
  File "", line 10, in remove_archetypes
    portal = api.portal.get()
  File "/opt/plone5.2.5_python2.7.clean/buildout-cache/eggs/cp27mu/plone.api-1.11.0-py2.7.egg/plone/api/", line 82, in get
    'Unable to get the portal object. More info on '
plone.api.exc.CannotGetPortalError: Unable to get the portal object. More info on

I looked at Portal — Plone Documentation v5.2 and it looks like portal = api.portal.get() is correct. My Plone instance has two sites in it /Main and /SchuhTest.

I also tried

portal = api.portal.get('Main')


portal = api['Main']

Do I need to do something to select one of these or something else?

@yurj I did some more searching and figure out that adding -O Main for my /Main site to the bin/instance run command solves the plone.api.exc.CannotGetPortalError.

Where do the messages go? I did not find them in var/log/instance.log. I added print for the lines and see this when I run

sudo -u plone_daemon bin/instance -O Main run 
Deleted portal_languages
Deleted portal_tinymce
Deleted kupu_library_tool
Deleted portal_factory
Deleted portal_atct
Deleted uid_catalog
Deleted archetype_tool
Deleted reference_catalog
Deleted portal_metadata

I keep seeing the same output when I rerun the script which tells me that the database on the disk is not being updated. Is there a way to run in a way that updates the Data.fs database file or some code I can add to to write the changes to Data.fs before exiting? Thanks again for the help. Michael

@yurj after some more searching around, I found adding import transaction to the beginning of and transaction.commit() at the end updates my var/Data.fs database file.

Even though the documentation says that INFO is the default logging level, I tried adding event-log-level = debug to [instance] in buildout.cfg and running buildout to see if that would help. Unfortunately, I am still not finding the messages written with or any other logging information when I run sudo -u plone_daemon bin/instance -O Main run Here is the [instance] part of my buildout.cfg file.

<= instance_base
recipe = plone.recipe.zope2instance
http-address = 8525
event-log-level = debug

I am using Plone 5.2.5. Where are the messages going? It appears from the documentation that I have read that they should be in the var/log/instance.log file but for some reason they are not going there. What am I doing wrong? Thanks.

1 Like

to log with "bin/instance run" scripts I once did something like this:

import logging
import sys

# sync STDOUT logger
sync_logger = logging.getLogger("my.package")
handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter("%(asctime)s %(levelname)s %(message)s")

and then you can use"Log something")

which should be shown on STDOUT ...

@petschki Awesome! Following what you wrote in your response, I add the 5 lines to set up the logger to show log messages on STDOUT and use the existing calls in Here is what I added:

log = logging.getLogger(__name__)

handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter("%(asctime)s %(levelname)s %(message)s")

Here is what I see when I run with my site SchuhTest.

sudo -u plone_daemon bin/instance -O SchuhTest run
2021-11-23 12:20:27,849 INFO Tool portal_languages not found
INFO:main:Tool portal_languages not found
2021-11-23 12:20:27,850 INFO Tool portal_tinymce not found
INFO:main:Tool portal_tinymce not found
2021-11-23 12:20:27,850 INFO Tool kupu_library_tool not found
INFO:main:Tool kupu_library_tool not found
2021-11-23 12:20:27,850 INFO Tool portal_factory not found
INFO:main:Tool portal_factory not found
2021-11-23 12:20:27,851 INFO Tool portal_atct not found
INFO:main:Tool portal_atct not found
2021-11-23 12:20:27,851 INFO Tool uid_catalog not found
INFO:main:Tool uid_catalog not found
2021-11-23 12:20:27,851 INFO Tool archetype_tool not found
INFO:main:Tool archetype_tool not found
2021-11-23 12:20:27,851 INFO Tool reference_catalog not found
INFO:main:Tool reference_catalog not found
2021-11-23 12:20:27,852 INFO Tool portal_metadata not found
INFO:main:Tool portal_metadata not found
2021-11-23 12:20:27,852 INFO fix_conversations:

It appears that the messages are showing up twice for some reason. It would be nice to fix it so that the messages only show up once, but having them displayed twice is a lot more helpful than not at all.
From reading log messages appearing twice with Python Logging - Stack Overflow and other pages, it appears that the logging.getLogger(__main__) call is setting up a second logger. I was not able to figure out how to update to display the logs only once. While it would be nice to fix the duplicate logging, twice is fine.

From searching for documentation, I found the Mastering Plone 5 Development — Plone Training 2021 documentation pages. It looks like it has a lot of good information in this training class.

Thank you @yurj and @petschki for helping me with this!

1 Like

since you use -O parameter I'd try to setup your portal with something like

from zope.component.hooks import setSite

setSite(obj)  # obj should be your portal object

before accessing anything further.

here are also a few things about commandline scripts which helped me out. maybe slightly outdated now though ... Command-line interaction and scripting — Plone Documentation v5.2

1 Like