Over the holidays I migrated the very small website https://www.stiftung-bayerische-gedenkstaetten.de from Plone 4.3 with Archetypes and LinguaPlone to Plone 5.2 with Dexterity on Python 3.8.
The project had 2 interesting aspects to it:
The Migration
The migration was finished rather quick in less than 2 days. The site only had a small number of addons that mostly had become obsolete for the project: Products.ImageEditor, webcouturier.dropdownmenu, collective.flowplayer plus custom policy and theme-packages. For small to medium projects I prefer to have only one package (in this case stiftung.site that holds everything)
Since the additional languages were never published (the usual story: the client never had enough time to deal with the additional work) I made it one-language only now. I will show the code at the end of this post. Most of it was taken from https://github.com/collective/collective.migrationhelpers with only small additions.
The process is the usual:
- Remove obsolete Addons, cleanup and remove LinguaPlone in Plone 4.3
- Upgrade to Plone 5.2
- Migrate to Dexterity
- Migrate to Python 3
The migration to Python 3 was nothing more than running
./bin/zodbupdate --convert-py3 --file=var/filestorage/Data.fs --encoding=utf8
The Theme
The old theme was done very old school by overwriting the sunburst theme. For the new theme I decided to try the same approach with barceloneta. The whole theme is only 69 css-rules that overwrite barceloneta, mostly the navigation and the portlets. Two problems coule not be solves with css only:
- The grid: column-one (that holds the left portlets) needs a width of 4 instead of thge default 3.
- The main-navigation needs to be in the header-container.
So I made a ttw-copy of the barceloneta-theme and modified two things:
- In
index.htmlI moved mainnavigation-wrapper up into the header with a width ofcol-md-8 - In
rules.xmlI changed the conditionally modified css-class ofportal-column-onetocol-xs-12 col-sm-12 col-md-8 col-md-push-4(indead ofcol-md-8 col-md-push-4).
The portal-header got its class col-md-4 by overriding plone.app.layout.viewlets.portal_header.pt with jbot.
I like the approach to keep modifications as small as possible because that makes the next upgrade easier. The whole package has almost no code, some useful registry settings and a bit of css.
The copy of the theme in the DB is bothering me a little but since the diff is so small (and I stored it diff) it will be easy to upgrade to future version of barceloneta. I would have preferred to only overwrite rules.xml and index.html in my package but that is not possible.
It was done quick but not very dirty and deployed on December 30, 2019.
Code
Here is the migration-code run on Plone 4.3:
# -*- coding: utf-8 -*-
from plone import api
from plone.portlets.interfaces import IPortletAssignmentMapping
from plone.portlets.interfaces import IPortletManager
from zope.component import getMultiAdapter
from zope.component import getUtility
from zExceptions import BadRequest
import logging
import transaction
log = logging.getLogger(__name__)
def prepare_plone5_upgrade(setup):
remove_overrides()
release_all_webdav_locks()
remove_all_revisions()
disable_theme()
portal = api.portal.get()
catalog = api.portal.get_tool('portal_catalog')
qi = api.portal.get_tool('portal_quickinstaller')
portal_skins = api.portal.get_tool('portal_skins')
portal_properties = api.portal.get_tool('portal_properties')
to_delete = [
'/de/service/news/max-mannheimer-videos/nie-wieder',
'/index_html',
'/en',
]
for path in to_delete:
try:
obj = api.content.get(path=path)
if obj is not None:
api.content.delete(obj, check_linkintegrity=False)
log.info('Deleted %s' % path)
except:
continue
# Remove webcouturier.dropdownmenu
if qi.isProductInstalled('webcouturier.dropdownmenu'):
qi.uninstallProducts(['webcouturier.dropdownmenu'])
# remove Products.ImageEditor
log.info('removing Products.ImageEditor')
if qi.isProductInstalled('ImageEditor'):
qi.uninstallProducts(['ImageEditor'])
# wildcard.media
log.info('removing wildcard.media')
if qi.isProductInstalled('wildcard.media'):
qi.uninstallProducts(['wildcard.media'])
# collective.flowplayer
log.info('removing collective.flowplayer')
if qi.isProductInstalled('collective.flowplayer'):
qi.uninstallProducts(['collective.flowplayer'])
# collective.js.fancybox
if qi.isProductInstalled('collective.js.fancybox'):
qi.uninstallProducts(['collective.js.fancybox'])
# collective.revisionmanager
if qi.isProductInstalled('collective.revisionmanager'):
qi.uninstallProducts(['collective.revisionmanager'])
log.info('rebuilding catalog')
catalog.clearFindAndRebuild()
manager = getUtility(
IPortletManager, name='plone.leftcolumn', context=portal)
mapping = getMultiAdapter((portal, manager), IPortletAssignmentMapping)
for mapping_id in mapping:
try:
del mapping[mapping_id]
except KeyError:
pass
log.info('Set languages')
for brain in catalog.unrestrictedSearchResults():
obj = brain.getObject()
if obj.Language() == 'en':
log.info('Deleteting english content %s' % obj.absolute_url())
api.content.delete(obj, check_linkintegrity=False)
elif obj.Language() != 'de':
obj.setLanguage('de')
obj.reindexObject(idxs=['Language'])
if obj.portal_type == 'WildcardVideo':
log.info('Deleteting %s' % obj.absolute_url())
api.content.delete(obj, check_linkintegrity=False)
transaction.commit()
log.info('remove LP')
if qi.isProductInstalled('LinguaPlone'):
qi.uninstallProducts(['LinguaPlone'])
try:
portal_properties.manage_delObjects(['linguaplone_properties'])
except BadRequest:
pass
transaction.commit()
log.info('run uninstall self')
# run our uninstall-profile that also removes LP
setup.runAllImportStepsFromProfile(
'profile-plonetheme.stiftung:uninstall', purge_old=False)
log.info('unregister_broken_persistent_components')
unregister_broken_persistent_components(portal)
log.info('remove_vocabularies')
remove_vocabularies(setup)
transaction.commit()
def remove_vocabularies(setup):
from plone.i18n.locales.interfaces import IContentLanguageAvailability
from plone.i18n.locales.interfaces import IMetadataLanguageAvailability
portal = api.portal.get()
sm = portal.getSiteManager()
if IContentLanguageAvailability in sm.utilities._subscribers[0]:
del sm.utilities._subscribers[0][IContentLanguageAvailability]
log.info(u'Unregistering subscriber for IContentLanguageAvailability')
if IMetadataLanguageAvailability in sm.utilities._subscribers[0]:
del sm.utilities._subscribers[0][IMetadataLanguageAvailability]
log.info(u'Unregistering subscriber for IMetadataLanguageAvailability')
if IMetadataLanguageAvailability in sm.utilities._adapters[0]:
del sm.utilities._adapters[0][IMetadataLanguageAvailability]
log.info(u'Unregistering adapter for IMetadataLanguageAvailability')
if IContentLanguageAvailability in sm.utilities._adapters[0]:
del sm.utilities._adapters[0][IContentLanguageAvailability]
log.info(u'Unregistering adapter for IContentLanguageAvailability')
sm.utilities._p_changed = True
def unregister_broken_persistent_components(portal):
sm = portal.getSiteManager()
for item in sm._utility_registrations.items():
if hasattr(item[1][0], '__Broken_state__'):
# unregisterUtility(component, provided, name)
# See: five.localsitemanager.registry.PersistentComponents.unregisterUtility # noqa: E501
log.info(u"Unregistering component {0}".format(item))
sm.unregisterUtility(item[1][0], item[0][0], item[0][1])
def remove_overrides(context=None):
log.info('removing portal_skins overrides')
portal_skins = api.portal.get_tool('portal_skins')
custom = portal_skins['custom']
for name in custom.keys():
custom.manage_delObjects([name])
log.info(u'Removed skin item {}'.format(name))
log.info('removing portal_view_customizations')
view_customizations = api.portal.get_tool('portal_view_customizations')
for name in view_customizations.keys():
view_customizations.manage_delObjects([name])
log.info(u'Removed portal_view_customizations item {}'.format(name))
def release_all_webdav_locks(context=None):
from Products.CMFPlone.utils import base_hasattr
portal = api.portal.get()
def unlock(obj, path):
if base_hasattr(obj, 'wl_isLocked') and obj.wl_isLocked():
obj.wl_clearLocks()
log.info(u'Unlocked {}'.format(path))
portal.ZopeFindAndApply(portal, search_sub=True, apply_func=unlock)
def remove_all_revisions(context=None):
"""Remove all revisions.
After packing the DB this could significantly shrink its size.
"""
hs = api.portal.get_tool('portal_historiesstorage')
zvcr = hs.zvc_repo
zvcr._histories.clear()
storage = hs._shadowStorage
storage._storage.clear()
def disable_theme(context=None):
"""Disable a custom diazo theme and enable sunburst.
Useful for cleaning up a site in Plone 4
"""
THEME_NAME = 'plonetheme.stiftung'
from plone.app.theming.utils import applyTheme
portal_skins = api.portal.get_tool('portal_skins')
qi = api.portal.get_tool('portal_quickinstaller')
if qi.isProductInstalled(THEME_NAME):
log.info('Uninstalling {}'.format(THEME_NAME))
qi.uninstallProducts([THEME_NAME])
log.info('Disabling all diazo themes')
# applyTheme(None)
log.info('Enabled Sunburst Theme')
portal_skins.default_skin = 'Sunburst Theme'
if THEME_NAME in portal_skins.getSkinSelections():
portal_skins.manage_skinLayers([THEME_NAME], del_skin=True)
Here is the migration-code run on Plone 5.2 (on Python 2):
def prep_dx_migration(context=None):
portal = api.portal.get()
request = getRequest()
installer = api.content.get_view('installer', portal, request)
if not installer.is_product_installed('plone.app.contenttypes'):
installer.install_product('plone.app.contenttypes')
def migrate_to_dexterity(setup):
portal = api.portal.get()
request = getRequest()
pac_migration = api.content.get_view('migrate_from_atct', portal, request)
pac_migration(
migrate=True,
migrate_schemaextended_content=True,
reindex_catalog=True,
)
def cleanup_after_migration_to_p5(setup):
portal = api.portal.get()
request = getRequest()
installer = api.content.get_view('installer', portal, request)
portal.setDefaultPage('die-stiftung')
addons = [
'Archtypes',
'ATContentTypes',
'plone.app.referenceablebehavior',
'plone.app.blob',
'plone.app.imaging',
]
for addon in addons:
if installer.is_product_installed(addon):
installer.uninstall_product(addon)
portal.manage_delObjects(['portal_atct'])
portal.manage_delObjects(['portal_factory'])
portal.manage_delObjects(['portal_languages'])
portal.manage_delObjects(['portal_metadata'])
portal.manage_delObjects(['portal_tinymce'])
portal._delObject('archetype_tool', suppress_events=True)
portal._delObject('reference_catalog', suppress_events=True)
portal._delObject('uid_catalog', suppress_events=True)
portal_properties = portal.portal_properties
portal_properties.manage_delObjects(['dropdown_properties'])
portal_properties.manage_delObjects(['imageeditor'])
portal_properties.manage_delObjects(['imaging_properties'])
if not installer.is_product_installed('plonetheme.barceloneta'):
installer.install_product('plonetheme.barceloneta')
broken_import_steps = [
u'collective.z3cform.datetimewidget',
u'wildcard.media.uninstall',
u'wildcard.media.install',
u'languagetool',
u'Products.ImageEditor.uninstall',
u'Products.ImageEditor.install',
]
registry = setup.getImportStepRegistry()
for broken_import_step in broken_import_steps:
if broken_import_step in registry.listSteps():
registry.unregisterStep(broken_import_step)
def cleanup_after_at_removal(setup):
portal = api.portal.get()
unregister_broken_persistent_components(portal)
def unregister_broken_persistent_components(portal):
sm = portal.getSiteManager()
for item in sm._utility_registrations.items():
if hasattr(item[1][0], '__Broken_state__'):
# unregisterUtility(component, provided, name)
# See: five.localsitemanager.registry.PersistentComponents.unregisterUtility # noqa: E501
log.info(u"Unregistering component {0}".format(item))
sm.unregisterUtility(item[1][0], item[0][0], item[0][1])
from Products.ATContentTypes.interfaces.interfaces import IATCTTool
from Products.CMFCore.interfaces import IMetadataTool
portal = api.portal.get()
sm = portal.getSiteManager()
if IMetadataTool in sm.utilities._subscribers[0]:
del sm.utilities._subscribers[0][IMetadataTool]
log.info(u'Unregistering subscriber for IMetadataTool')
if IATCTTool in sm.utilities._subscribers[0]:
del sm.utilities._subscribers[0][IATCTTool]
log.info(u'Unregistering subscriber for IATCTTool')
if IATCTTool in sm.utilities._adapters[0]:
del sm.utilities._adapters[0][IATCTTool]
log.info(u'Unregistering adapter for IATCTTool')
if IMetadataTool in sm.utilities._adapters[0]:
del sm.utilities._adapters[0][IMetadataTool]
log.info(u'Unregistering adapter for IMetadataTool')
if IATCTTool in sm.utilities._provided:
del sm.utilities._provided[IATCTTool]
log.info(u'Unregistering _provided for IATCTTool')
if IMetadataTool in sm.utilities._provided:
del sm.utilities._provided[IMetadataTool]
log.info(u'Unregistering _provided for IMetadataTool')
sm.utilities._p_changed = True
def restructure_content(context=None):
portal = api.portal.get()
request = getRequest()
de = portal['de']
for key in de:
api.content.move(source=de[key], target=portal)
api.content.delete(de)
portal.setDefaultPage('die-stiftung')
manage_viewlets = api.content.get_view('manage-viewlets', portal, request)
# hide search viewlet
manage_viewlets.hide('plone.portalheader', 'plone.searchbox')
manage_viewlets.hide('plone.mainnavigation', 'plone.path_bar')
# Move leadimage above breadcrumbs
try:
manage_viewlets.moveAbove(
'plone.mainnavigation', 'plone.path_bar', 'plone.global_sections'
)
except ValueError:
pass