Small project migration with customized barceloneta

Over the holidays I migrated the very small website https://www.stiftung-bayerische-gedenkstaetten.de from Plone 4.3 with Archetypes and LinguaPlone to Plone 5.2 with Dexterity on Python 3.8.

The project had 2 interesting aspects to it:

The Migration

The migration was finished rather quick in less than 2 days. The site only had a small number of addons that mostly had become obsolete for the project: Products.ImageEditor, webcouturier.dropdownmenu, collective.flowplayer plus custom policy and theme-packages. For small to medium projects I prefer to have only one package (in this case stiftung.site that holds everything)
Since the additional languages were never published (the usual story: the client never had enough time to deal with the additional work) I made it one-language only now. I will show the code at the end of this post. Most of it was taken from https://github.com/collective/collective.migrationhelpers with only small additions.

The process is the usual:

  • Remove obsolete Addons, cleanup and remove LinguaPlone in Plone 4.3
  • Upgrade to Plone 5.2
  • Migrate to Dexterity
  • Migrate to Python 3

The migration to Python 3 was nothing more than running

./bin/zodbupdate --convert-py3 --file=var/filestorage/Data.fs --encoding=utf8

The Theme

The old theme was done very old school by overwriting the sunburst theme. For the new theme I decided to try the same approach with barceloneta. The whole theme is only 69 css-rules that overwrite barceloneta, mostly the navigation and the portlets. Two problems coule not be solves with css only:

  • The grid: column-one (that holds the left portlets) needs a width of 4 instead of thge default 3.
  • The main-navigation needs to be in the header-container.

So I made a ttw-copy of the barceloneta-theme and modified two things:

  • In index.html I moved mainnavigation-wrapper up into the header with a width of col-md-8
  • In rules.xml I changed the conditionally modified css-class of portal-column-one to col-xs-12 col-sm-12 col-md-8 col-md-push-4 (indead of col-md-8 col-md-push-4).

The portal-header got its class col-md-4 by overriding plone.app.layout.viewlets.portal_header.pt with jbot.

I like the approach to keep modifications as small as possible because that makes the next upgrade easier. The whole package has almost no code, some useful registry settings and a bit of css.
The copy of the theme in the DB is bothering me a little but since the diff is so small (and I stored it diff) it will be easy to upgrade to future version of barceloneta. I would have preferred to only overwrite rules.xml and index.html in my package but that is not possible.

It was done quick but not very dirty and deployed on December 30, 2019.

Code

Here is the migration-code run on Plone 4.3:

# -*- coding: utf-8 -*-
from plone import api
from plone.portlets.interfaces import IPortletAssignmentMapping
from plone.portlets.interfaces import IPortletManager
from zope.component import getMultiAdapter
from zope.component import getUtility
from zExceptions import BadRequest

import logging
import transaction

log = logging.getLogger(__name__)


def prepare_plone5_upgrade(setup):
    remove_overrides()
    release_all_webdav_locks()
    remove_all_revisions()
    disable_theme()
    portal = api.portal.get()
    catalog = api.portal.get_tool('portal_catalog')
    qi = api.portal.get_tool('portal_quickinstaller')
    portal_skins = api.portal.get_tool('portal_skins')
    portal_properties = api.portal.get_tool('portal_properties')

    to_delete = [
        '/de/service/news/max-mannheimer-videos/nie-wieder',
        '/index_html',
        '/en',
    ]
    for path in to_delete:
        try:
            obj = api.content.get(path=path)
            if obj is not None:
                api.content.delete(obj, check_linkintegrity=False)
            log.info('Deleted %s' % path)
        except:
            continue

    # Remove webcouturier.dropdownmenu
    if qi.isProductInstalled('webcouturier.dropdownmenu'):
        qi.uninstallProducts(['webcouturier.dropdownmenu'])

    # remove Products.ImageEditor
    log.info('removing Products.ImageEditor')
    if qi.isProductInstalled('ImageEditor'):
        qi.uninstallProducts(['ImageEditor'])

    # wildcard.media
    log.info('removing wildcard.media')
    if qi.isProductInstalled('wildcard.media'):
        qi.uninstallProducts(['wildcard.media'])

    # collective.flowplayer
    log.info('removing collective.flowplayer')
    if qi.isProductInstalled('collective.flowplayer'):
        qi.uninstallProducts(['collective.flowplayer'])

    # collective.js.fancybox
    if qi.isProductInstalled('collective.js.fancybox'):
        qi.uninstallProducts(['collective.js.fancybox'])

    # collective.revisionmanager
    if qi.isProductInstalled('collective.revisionmanager'):
        qi.uninstallProducts(['collective.revisionmanager'])

    log.info('rebuilding catalog')
    catalog.clearFindAndRebuild()

    manager = getUtility(
        IPortletManager, name='plone.leftcolumn', context=portal)
    mapping = getMultiAdapter((portal, manager), IPortletAssignmentMapping)
    for mapping_id in mapping:
        try:
            del mapping[mapping_id]
        except KeyError:
            pass

    log.info('Set languages')
    for brain in catalog.unrestrictedSearchResults():
        obj = brain.getObject()
        if obj.Language() == 'en':
            log.info('Deleteting english content %s' % obj.absolute_url())
            api.content.delete(obj, check_linkintegrity=False)
        elif obj.Language() != 'de':
            obj.setLanguage('de')
            obj.reindexObject(idxs=['Language'])
        if obj.portal_type == 'WildcardVideo':
            log.info('Deleteting %s' % obj.absolute_url())
            api.content.delete(obj, check_linkintegrity=False)
    transaction.commit()

    log.info('remove LP')
    if qi.isProductInstalled('LinguaPlone'):
        qi.uninstallProducts(['LinguaPlone'])
    try:
        portal_properties.manage_delObjects(['linguaplone_properties'])
    except BadRequest:
        pass

    transaction.commit()

    log.info('run uninstall self')
    # run our uninstall-profile that also removes LP
    setup.runAllImportStepsFromProfile(
        'profile-plonetheme.stiftung:uninstall', purge_old=False)

    log.info('unregister_broken_persistent_components')
    unregister_broken_persistent_components(portal)
    log.info('remove_vocabularies')
    remove_vocabularies(setup)
    transaction.commit()


def remove_vocabularies(setup):
    from plone.i18n.locales.interfaces import IContentLanguageAvailability
    from plone.i18n.locales.interfaces import IMetadataLanguageAvailability
    portal = api.portal.get()
    sm = portal.getSiteManager()

    if IContentLanguageAvailability in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IContentLanguageAvailability]
        log.info(u'Unregistering subscriber for IContentLanguageAvailability')
    if IMetadataLanguageAvailability in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IMetadataLanguageAvailability]
        log.info(u'Unregistering subscriber for IMetadataLanguageAvailability')

    if IMetadataLanguageAvailability in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IMetadataLanguageAvailability]
        log.info(u'Unregistering adapter for IMetadataLanguageAvailability')
    if IContentLanguageAvailability in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IContentLanguageAvailability]
        log.info(u'Unregistering adapter for IContentLanguageAvailability')

    sm.utilities._p_changed = True


def unregister_broken_persistent_components(portal):
    sm = portal.getSiteManager()

    for item in sm._utility_registrations.items():
        if hasattr(item[1][0], '__Broken_state__'):
            # unregisterUtility(component, provided, name)
            # See: five.localsitemanager.registry.PersistentComponents.unregisterUtility  # noqa: E501
            log.info(u"Unregistering component {0}".format(item))
            sm.unregisterUtility(item[1][0], item[0][0], item[0][1])


def remove_overrides(context=None):
    log.info('removing portal_skins overrides')
    portal_skins = api.portal.get_tool('portal_skins')
    custom = portal_skins['custom']
    for name in custom.keys():
        custom.manage_delObjects([name])
        log.info(u'Removed skin item {}'.format(name))

    log.info('removing portal_view_customizations')
    view_customizations = api.portal.get_tool('portal_view_customizations')
    for name in view_customizations.keys():
        view_customizations.manage_delObjects([name])
        log.info(u'Removed portal_view_customizations item {}'.format(name))


def release_all_webdav_locks(context=None):
    from Products.CMFPlone.utils import base_hasattr
    portal = api.portal.get()

    def unlock(obj, path):
        if base_hasattr(obj, 'wl_isLocked') and obj.wl_isLocked():
            obj.wl_clearLocks()
            log.info(u'Unlocked {}'.format(path))

    portal.ZopeFindAndApply(portal, search_sub=True, apply_func=unlock)


def remove_all_revisions(context=None):
    """Remove all revisions.
    After packing the DB this could significantly shrink its size.
    """
    hs = api.portal.get_tool('portal_historiesstorage')
    zvcr = hs.zvc_repo
    zvcr._histories.clear()
    storage = hs._shadowStorage
    storage._storage.clear()


def disable_theme(context=None):
    """Disable a custom diazo theme and enable sunburst.
    Useful for cleaning up a site in Plone 4
    """
    THEME_NAME = 'plonetheme.stiftung'
    from plone.app.theming.utils import applyTheme
    portal_skins = api.portal.get_tool('portal_skins')
    qi = api.portal.get_tool('portal_quickinstaller')
    if qi.isProductInstalled(THEME_NAME):
        log.info('Uninstalling {}'.format(THEME_NAME))
        qi.uninstallProducts([THEME_NAME])
    log.info('Disabling all diazo themes')
    # applyTheme(None)
    log.info('Enabled Sunburst Theme')
    portal_skins.default_skin = 'Sunburst Theme'
    if THEME_NAME in portal_skins.getSkinSelections():
        portal_skins.manage_skinLayers([THEME_NAME], del_skin=True)

Here is the migration-code run on Plone 5.2 (on Python 2):


def prep_dx_migration(context=None):
    portal = api.portal.get()
    request = getRequest()
    installer = api.content.get_view('installer', portal, request)
    if not installer.is_product_installed('plone.app.contenttypes'):
        installer.install_product('plone.app.contenttypes')


def migrate_to_dexterity(setup):
    portal = api.portal.get()
    request = getRequest()
    pac_migration = api.content.get_view('migrate_from_atct', portal, request)
    pac_migration(
        migrate=True,
        migrate_schemaextended_content=True,
        reindex_catalog=True,
    )


def cleanup_after_migration_to_p5(setup):
    portal = api.portal.get()
    request = getRequest()
    installer = api.content.get_view('installer', portal, request)
    portal.setDefaultPage('die-stiftung')
    addons = [
        'Archtypes',
        'ATContentTypes',
        'plone.app.referenceablebehavior',
        'plone.app.blob',
        'plone.app.imaging',
    ]
    for addon in addons:
        if installer.is_product_installed(addon):
            installer.uninstall_product(addon)
    portal.manage_delObjects(['portal_atct'])
    portal.manage_delObjects(['portal_factory'])
    portal.manage_delObjects(['portal_languages'])
    portal.manage_delObjects(['portal_metadata'])
    portal.manage_delObjects(['portal_tinymce'])
    portal._delObject('archetype_tool', suppress_events=True)
    portal._delObject('reference_catalog', suppress_events=True)
    portal._delObject('uid_catalog', suppress_events=True)

    portal_properties = portal.portal_properties
    portal_properties.manage_delObjects(['dropdown_properties'])
    portal_properties.manage_delObjects(['imageeditor'])
    portal_properties.manage_delObjects(['imaging_properties'])

    if not installer.is_product_installed('plonetheme.barceloneta'):
        installer.install_product('plonetheme.barceloneta')

    broken_import_steps = [
        u'collective.z3cform.datetimewidget',
        u'wildcard.media.uninstall',
        u'wildcard.media.install',
        u'languagetool',
        u'Products.ImageEditor.uninstall',
        u'Products.ImageEditor.install',
    ]
    registry = setup.getImportStepRegistry()
    for broken_import_step in broken_import_steps:
        if broken_import_step in registry.listSteps():
            registry.unregisterStep(broken_import_step)


def cleanup_after_at_removal(setup):
    portal = api.portal.get()
    unregister_broken_persistent_components(portal)


def unregister_broken_persistent_components(portal):
    sm = portal.getSiteManager()

    for item in sm._utility_registrations.items():
        if hasattr(item[1][0], '__Broken_state__'):
            # unregisterUtility(component, provided, name)
            # See: five.localsitemanager.registry.PersistentComponents.unregisterUtility  # noqa: E501
            log.info(u"Unregistering component {0}".format(item))
            sm.unregisterUtility(item[1][0], item[0][0], item[0][1])

    from Products.ATContentTypes.interfaces.interfaces import IATCTTool
    from Products.CMFCore.interfaces import IMetadataTool
    portal = api.portal.get()
    sm = portal.getSiteManager()

    if IMetadataTool in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IMetadataTool]
        log.info(u'Unregistering subscriber for IMetadataTool')
    if IATCTTool in sm.utilities._subscribers[0]:
        del sm.utilities._subscribers[0][IATCTTool]
        log.info(u'Unregistering subscriber for IATCTTool')

    if IATCTTool in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IATCTTool]
        log.info(u'Unregistering adapter for IATCTTool')
    if IMetadataTool in sm.utilities._adapters[0]:
        del sm.utilities._adapters[0][IMetadataTool]
        log.info(u'Unregistering adapter for IMetadataTool')

    if IATCTTool in sm.utilities._provided:
        del sm.utilities._provided[IATCTTool]
        log.info(u'Unregistering _provided for IATCTTool')
    if IMetadataTool in sm.utilities._provided:
        del sm.utilities._provided[IMetadataTool]
        log.info(u'Unregistering _provided for IMetadataTool')

    sm.utilities._p_changed = True


def restructure_content(context=None):
    portal = api.portal.get()
    request = getRequest()

    de = portal['de']
    for key in de:
        api.content.move(source=de[key], target=portal)
    api.content.delete(de)
    portal.setDefaultPage('die-stiftung')

    manage_viewlets = api.content.get_view('manage-viewlets', portal, request)
    # hide search viewlet
    manage_viewlets.hide('plone.portalheader', 'plone.searchbox')
    manage_viewlets.hide('plone.mainnavigation', 'plone.path_bar')

    # Move leadimage above breadcrumbs
    try:
        manage_viewlets.moveAbove(
            'plone.mainnavigation', 'plone.path_bar', 'plone.global_sections'
        )
    except ValueError:
        pass
12 Likes

Plone Foundation Code of Conduct