Upgrading/Migrating from 5.1.2 to 5.2

ronc · October 15, 2019, 2:11am

Fear not, I"m trying this is on a development server first...

This isn't likely the best upgrade path, but I thought I would give it a whirl. I took a perfectly functional Data.fs* set from a 5.1.2 site to a newly installed 5.2 site. I was greeted with the opportunity to upgrade. I hoped for the best. Partial success, I think, but ultimately, it did not work. Here is the result for posterity. Incidentally, this is a stock Plone site (both 5.1.2 and 5.2). Any help with the migration would be greatly appreciated.

And without further ado...

Upgrade report

Starting the migration from version: 5112
Role / permission map imported.
Removed record plone.resources/resource-plone-app-event-js.deps.
Removed record plone.resources/resource-plone-app-event-js.url.
Removed record plone.resources/resource-plone-app-event-js.js.
Removed record plone.resources/resource-plone-app-event-js.init.
Removed record plone.resources/resource-plone-app-event-js.export.
Removed record plone.resources/resource-plone-app-event-js.conf.
Removed record plone.resources/resource-plone-app-event-js.css.
Removed record plone.resources/resource-plone-app-event-css.deps.
Removed record plone.resources/resource-plone-app-event-css.url.
Removed record plone.resources/resource-plone-app-event-css.js.
Removed record plone.resources/resource-plone-app-event-css.init.
Removed record plone.resources/resource-plone-app-event-css.export.
Removed record plone.resources/resource-plone-app-event-css.conf.
Removed record plone.resources/resource-plone-app-event-css.css.
Ran upgrade step: Run to513 upgrade profile.
Ran upgrade step: Force remove old p.a.event resources
Role / permission map imported.
Removed record plone.resources/jquery-highlightsearchterms.deps.
Removed record plone.resources/jquery-highlightsearchterms.url.
Removed record plone.resources/jquery-highlightsearchterms.js.
Removed record plone.resources/jquery-highlightsearchterms.init.
Removed record plone.resources/jquery-highlightsearchterms.export.
Removed record plone.resources/jquery-highlightsearchterms.conf.
Removed record plone.resources/jquery-highlightsearchterms.css.
Ran upgrade step: Run to514 upgrade profile.
Ran upgrade step: Miscellaneous
Role / permission map imported.
Ran upgrade step: Run to515 upgrade profile.
Ran upgrade step: Miscellaneous
Role / permission map imported.
Adapters registered.
The object None was not found, while trying to register an utility. The provided object definition was portal_metadata. The site used was: <PloneSite at /rlc>
The object None was not found, while trying to register an utility. The provided object definition was portal_syndication. The site used was: <PloneSite at /rlc>
The object None was not found, while trying to register an utility. The provided object definition was portal_undo. The site used was: <PloneSite at /rlc>
Utilities registered.
Skins tool imported.
Removed record plone.resources/jquery-highlightsearchterms.deps.
Removed record plone.resources/jquery-highlightsearchterms.url.
Removed record plone.resources/jquery-highlightsearchterms.js.
Removed record plone.resources/jquery-highlightsearchterms.init.
Removed record plone.resources/jquery-highlightsearchterms.export.
Removed record plone.resources/jquery-highlightsearchterms.conf.
Removed record plone.resources/jquery-highlightsearchterms.css.
Rebuilding member data information. This step can take a while if your site has many users.
Ran upgrade step: Run to52alpha1 upgrade profile.
Ran upgrade step: Miscellaneous
Role / permission map imported.
Not creating required tool portal_css, because class Products.ResourceRegistries.tools.CSSRegistry.CSSRegistryTool is not found.
Not creating required tool portal_javascripts, because class Products.ResourceRegistries.tools.JSRegistry.JSRegistryTool is not found.
Toolset imported.
Actions tool imported.
Control panel imported.
RepositoryTool settings imported.
Removed record plone.resources/expect.deps.
Removed record plone.resources/expect.url.
Removed record plone.resources/expect.js.
Removed record plone.resources/expect.init.
Removed record plone.resources/expect.export.
Removed record plone.resources/expect.conf.
Removed record plone.resources/expect.css.
Removed record plone.resources/js-shortcuts.deps.
Removed record plone.resources/js-shortcuts.url.
Removed record plone.resources/js-shortcuts.js.
Removed record plone.resources/js-shortcuts.init.
Removed record plone.resources/js-shortcuts.export.
Removed record plone.resources/js-shortcuts.conf.
Removed record plone.resources/js-shortcuts.css.
Removed record plone.resources/marked.deps.
Removed record plone.resources/marked.url.
Removed record plone.resources/marked.js.
Removed record plone.resources/marked.init.
Removed record plone.resources/marked.export.
Removed record plone.resources/marked.conf.
Removed record plone.resources/marked.css.
Removed record plone.resources/rjs.deps.
Removed record plone.resources/rjs.url.
Removed record plone.resources/rjs.js.
Removed record plone.resources/rjs.init.
Removed record plone.resources/rjs.export.
Removed record plone.resources/rjs.conf.
Removed record plone.resources/rjs.css.
Removed record plone.resources/react.deps.
Removed record plone.resources/react.url.
Removed record plone.resources/react.js.
Removed record plone.resources/react.init.
Removed record plone.resources/react.export.
Removed record plone.resources/react.conf.
Removed record plone.resources/react.css.
Removed record plone.resources/JSXTransformer.deps.
Removed record plone.resources/JSXTransformer.url.
Removed record plone.resources/JSXTransformer.js.
Removed record plone.resources/JSXTransformer.init.
Removed record plone.resources/JSXTransformer.export.
Removed record plone.resources/JSXTransformer.conf.
Removed record plone.resources/JSXTransformer.css.
Removed record plone.resources/sinon.deps.
Removed record plone.resources/sinon.url.
Removed record plone.resources/sinon.js.
Removed record plone.resources/sinon.init.
Removed record plone.resources/sinon.export.
Removed record plone.resources/sinon.conf.
Removed record plone.resources/sinon.css.
Added BooleanIndex for field exclude_from_nav.
Indexing new indexes exclude_from_nav.
Cleaned up the toolset registry.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Broken relation removed.
Upgrade aborted. Error:
Traceback (most recent call last):
File "/opt/plone/buildout-cache/eggs/Products.CMFPlone-5.2.0-py2.7.egg/Products/CMFPlone/MigrationTool.py", line 291, in upgrade
step['step'].doStep(setup)
File "/opt/plone/buildout-cache/eggs/Products.GenericSetup-2.0-py2.7.egg/Products/GenericSetup/upgrade.py", line 168, in doStep
self.handler(tool)
File "/opt/plone/buildout-cache/eggs/plone.app.upgrade-2.0.27-py2.7.egg/plone/app/upgrade/v52/betas.py", line 110, in to52beta1
remove_interface_indexes_from_relations_catalog()
File "/opt/plone/buildout-cache/eggs/plone.app.upgrade-2.0.27-py2.7.egg/plone/app/upgrade/v52/betas.py", line 72, in remove_interface_indexes_from_relations_catalog
catalog.unindex(rel)
File "/opt/plone/buildout-cache/eggs/zc.relation-1.1.post2-py2.7.egg/zc/relation/catalog.py", line 564, in unindex
self.unindex_doc(self._relTools['dump'](rel, self, {}))
File "/opt/plone/buildout-cache/eggs/z3c.relationfield-0.8.0-py2.7.egg/z3c/relationfield/index.py", line 42, in dump
return intids.getId(obj)
File "/opt/plone/buildout-cache/eggs/five.intid-1.2.3-py2.7.egg/five/intid/intid.py", line 41, in getId
return z3IntIds.getId(self, ob)
File "/opt/plone/buildout-cache/eggs/zope.intid-4.3.0-py2.7.egg/zope/intid/init.py", line 108, in getId
raise IntIdMissingError(ob)
IntIdMissingError: <z3c.relationfield.relation.RelationValue object at 0x7f69d1aa6050 oid 0x85f9 in <Connection at 7f69d9b01d90>>
End of upgrade path, main migration has finished.
The upgrade path did NOT reach current version.
Migration has failed

Version Overview

Plone 5.2.0
Zope 4.1.1
Python 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]

mauritsvanrees · October 15, 2019, 4:10pm

You likes there is a relation between two objects, where at least one of them has not been registered yet with the 'intids' (integer ids) utility. Perhaps this script helps, going though all content to register intids.

scmos · October 16, 2019, 12:30am

Hello I have tried to upgrade from Plone 5.1.5 to 5.1.6 using "collective.cover" as a base and specifying version 5.1.6 as follow:

[buildout]
extends =

http://dist.plone.org/release/5.1.5/versions.cfg

http://dist.plone.org/release/5.1.6/versions.cfg

find-links =

http://dist.plone.org/release/5.1.5

http://dist.plone.org/release/5.1.6

And then run ~ /bin/buildout
And then run ~ /bin/instance fg
And then update the existing site in plone 5.1.5

And the screen shows me the error: "The upgrade path did NOT reach current version."

Also run the following script:
~/bin/instance run ~ /plonescripts/register_intids.py
and answer "No fixes were needed"

What am I doing wrong?

mauritsvanrees · October 19, 2019, 10:06pm

The terminal where you started bin/instance fg should show an error with a hopefully clear traceback. Is that the case? If the traceback or error is not clear to you, can you post it?

ronc · October 21, 2019, 7:13pm

:/opt/plone/zinstance/bin$ sudo -u plone_daemon ./instance fg start
/opt/plone/buildout-cache/eggs/Products.CMFUid-3.0.1-py2.7.egg/Products/CMFUid/UniqueIdHandlerTool.py:24: DeprecationWarning: InitializeClass is deprecated. Please import from AccessControl.class_init.
from App.class_init import InitializeClass
/opt/plone/buildout-cache/eggs/Products.CMFEditions-3.3.2-py2.7.egg/Products/CMFEditions/StandardModifiers.py:46: DeprecationWarning: ComponentLookupError is deprecated. Import from zope.interface.interfaces
from zope.component.interfaces import ComponentLookupError
/opt/plone/buildout-cache/eggs/Products.GenericSetup-2.0-py2.7.egg/Products/GenericSetup/registry.py:16: ImportWarning: Not importing directory '/opt/plone/buildout-cache/eggs/Products.GenericSetup-2.0-py2.7.egg/Products/GenericSetup/xml': missing init.py
from xml.sax import parseString
2019-10-21 14:12:29,951 WARNING [Init:89][MainThread] Class Products.CMFFormController.ControllerPythonScript.ControllerPythonScript has a security declaration for nonexistent method 'ZPythonScriptHTML_changePrefs'
2019-10-21 14:12:29,954 WARNING [Init:89][MainThread] Class Products.CMFFormController.ControllerValidator.ControllerValidator has a security declaration for nonexistent method 'ZPythonScriptHTML_changePrefs'
/opt/plone/buildout-cache/eggs/Products.CMFPlacefulWorkflow-2.0.0-py2.7.egg/Products/CMFPlacefulWorkflow/permissions.py:5: DeprecationWarning: setDefaultRoles is deprecated. Please use addPermission from AccessControl.Permission.
from Products.CMFCore.permissions import setDefaultRoles
/opt/plone/buildout-cache/eggs/plone.app.upgrade-2.0.27-py2.7.egg/plone/app/upgrade/init.py:167: DeprecationWarning: LockItem is deprecated. Please import from OFS.LockItem.
from webdav.LockItem import LockItem
/opt/plone/buildout-cache/eggs/plone.subrequest-1.9.1-py2.7.egg/plone/subrequest/init.py:18: DeprecationWarning: zope.site.hooks has moved to zope.component.hooks. Import of zope.site.hooks will become unsupported in Version 5.0
from zope.site.hooks import getSite
/opt/plone/buildout-cache/eggs/z3c.caching-2.1-py2.7.egg/z3c/caching/interfaces.py:2: DeprecationWarning: IObjectEvent is deprecated. Import from zope.interface.interfaces
from zope.component.interfaces import IObjectEvent
/opt/plone/buildout-cache/eggs/plone.app.content-3.7.0-py2.7.egg/plone/app/content/browser/vocabulary.py:10: DeprecationWarning: IFieldPermissionChecker is deprecated. Import IFieldPermissionChecker from plone.app.z3cform.interfaces instead
from plone.app.widgets.interfaces import IFieldPermissionChecker
/opt/plone/buildout-cache/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: IComponentRegistry is deprecated. Import from zope.interface.interfaces
obj = getattr(mod, oname)
/opt/plone/buildout-cache/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LockableItem is deprecated. Please import from OFS.Lockable.
obj = getattr(mod, oname)
/opt/plone/buildout-cache/eggs/plone.schemaeditor-2.0.20-py2.7.egg/plone/schemaeditor/utils.py:6: DeprecationWarning: ObjectEvent is deprecated. Import from zope.interface.interfaces
from zope.component.interfaces import ObjectEvent
/opt/plone/buildout-cache/eggs/plone.app.layout-3.2.1-py2.7.egg/plone/app/layout/viewlets/common.py:19: DeprecationWarning: ILanguageSchema is deprecated. It has been moved to plone.i18n.interfaces, import from there instead.
from Products.CMFPlone.interfaces.controlpanel import ILanguageSchema
/opt/plone/buildout-cache/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LazyMap is deprecated. Please import from ZTUtils.Lazy.
obj = getattr(mod, oname)
/opt/plone/buildout-cache/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LazyCat is deprecated. Please import from ZTUtils.Lazy.
obj = getattr(mod, oname)
/opt/plone/buildout-cache/eggs/plone.app.layout-3.2.1-py2.7.egg/plone/app/layout/globals/portal.py:11: DeprecationWarning: ILanguageSchema is deprecated. It has been moved to plone.i18n.interfaces, import from there instead.
from Products.CMFPlone.interfaces import ILanguageSchema
/opt/plone/buildout-cache/eggs/z3c.caching-2.1-py2.7.egg/z3c/caching/registry.py:16: DeprecationWarning: IComponents is deprecated. Import from zope.interface.interfaces
from zope.component.interfaces import IComponents
/opt/plone/buildout-cache/eggs/plone.portlet.collection-3.3.1-py2.7.egg/plone/portlet/collection/collection.py:2: DeprecationWarning: isDefaultPage is deprecated. Import from Products.CMFPlone instead
from plone.app.layout.navigation.defaultpage import isDefaultPage
/opt/plone/buildout-cache/eggs/Products.PluggableAuthService-2.0-py2.7.egg/Products/PluggableAuthService/plugins/exportimport.py:19: ImportWarning: Not importing directory '/opt/plone/buildout-cache/eggs/Products.PluggableAuthService-2.0-py2.7.egg/Products/PluggableAuthService/plugins/xml': missing init.py
from xml.dom.minidom import parseString
/opt/plone/buildout-cache/eggs/plone.restapi-4.3.1-py2.7.egg/plone/restapi/serializer/catalog.py:5: DeprecationWarning: Lazy is deprecated. Please import from ZTUtils.Lazy.
from Products.ZCatalog.Lazy import Lazy
2019-10-21 14:12:34,288 WARNING [plone.behavior:172][MainThread] Specifying 'for' in behavior 'Tiles' if no 'factory' is given has no effect and is superfluous.
/opt/plone/buildout-cache/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:225: ImportWarning: Not importing directory '/opt/plone/buildout-cache/eggs/plone.app.iterate-3.3.12-py2.7.egg/plone/app/iterate/profiles': missing init.py
import(mname)
2019-10-21 14:12:35,185 INFO [Zope:45][MainThread] Ready to handle requests
Starting server in PID 23063.
/opt/plone/buildout-cache/eggs/waitress-1.3.0-py2.7.egg/waitress/adjustments.py:432: DeprecationWarning: In future versions of Waitress clear_untrusted_proxy_headers will be set to True by default. You may opt-out by setting this value to False, or opt-in explicitly by setting this to True.
DeprecationWarning
Serving on http://0.0.0.0:8080

mauritsvanrees · October 22, 2019, 7:48am

You are missing an upgrade step. I see you are using plone.app.upgrade 2.0.27. For Plone 5.1.6 you need 2.0.28.

mauritsvanrees · October 22, 2019, 7:51am

It could be that you get the old version somehow because you are extending the versions cfg from http instead of https. The dist.plone.org used to serve both http and https, but currently http is redirected to https. Maybe that causes problems here.

Or check if you override the plone.app.upgrade version somewhere.

scmos · October 22, 2019, 7:40pm

(py27) scmos@scmoshp:~/scmos.elmirador01$ bin/instance fg
2019-10-22 16:37:18 INFO ZServer HTTP server started at Tue Oct 22 16:37:18 2019
Hostname: 0.0.0.0
Port: 8080
/home/scmos/scmos.elmirador01/eggs/plone.jsonserializer-0.9.6-py2.7.egg/plone/jsonserializer/serializer/converters.py:43: DeprecationWarning: getSiteEncoding: getSiteEncoding is deprecated. Plone only supports UTF-8 currently. This method always returns "utf-8"
from Products.CMFPlone.utils import getSiteEncoding
2019-10-22 16:37:21 INFO chameleon.config directory cache: /home/scmos/scmos.elmirador01/var/chameleon-cache.
/home/scmos/scmos.elmirador01/eggs/Products.CMFPlone-5.1.6-py2.7.egg/Products/CMFPlone/browser/syndication/views.py:17: DeprecationWarning: wrap_form is deprecated. Import from plone.z3cform.layout instead.
from plone.app.z3cform.layout import wrap_form
2019-10-22 16:37:24 WARNING fingerpointing common No audit log file specified; audit log view will be disabled
2019-10-22 16:37:24 WARNING plone.behavior Specifying 'for' in behavior 'Page refresh' if no 'factory' is given has no effect and is superfluous.
/home/scmos/scmos.elmirador01/eggs/collective.cover-2.2.0-py2.7.egg/collective/cover/tiles/file.py:11: DeprecationWarning: MimeTypeException is deprecated. Import from Products.MimetypesRegistry.interfaces instead
from Products.MimetypesRegistry.common import MimeTypeException
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.add.DefaultAddView has a security declaration for nonexistent method 'publishTraverse'
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.add.DefaultAddView has a security declaration for nonexistent method 'browserDefault'
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.edit.DefaultEditView has a security declaration for nonexistent method 'publishTraverse'
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.edit.DefaultEditView has a security declaration for nonexistent method 'browserDefault'
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.delete.DefaultDeleteView has a security declaration for nonexistent method 'publishTraverse'
2019-10-22 16:37:25 WARNING Init Class plone.app.tiles.browser.delete.DefaultDeleteView has a security declaration for nonexistent method 'browserDefault'
2019-10-22 16:37:25 WARNING Init Class collective.cover.tiles.configuration_view.DefaultConfigureView has a security declaration for nonexistent method 'publishTraverse'
2019-10-22 16:37:25 WARNING Init Class collective.cover.tiles.configuration_view.DefaultConfigureView has a security declaration for nonexistent method 'browserDefault'
2019-10-22 16:37:25 WARNING Init Class collective.cover.tiles.edit.CustomTileEdit has a security declaration for nonexistent method 'publishTraverse'
2019-10-22 16:37:25 WARNING Init Class collective.cover.tiles.edit.CustomTileEdit has a security declaration for nonexistent method 'browserDefault'
/home/scmos/scmos.elmirador01/eggs/Zope2-2.13.29-py2.7.egg/OFS/Application.py:102: DeprecationWarning: Expected text
transaction.get().note("Created Zope Application")
/home/scmos/scmos.elmirador01/eggs/Zope2-2.13.29-py2.7.egg/OFS/Application.py:267: DeprecationWarning: Expected text
transaction.get().note(note)
/home/scmos/scmos.elmirador01/eggs/Zope2-2.13.29-py2.7.egg/OFS/Application.py:523: DeprecationWarning: Expected text
transaction.get().note('Prior to product installs')
2019-10-22 16:37:25 INFO Zope Ready to handle requests
/home/scmos/scmos.elmirador01/eggs/Zope2-2.13.29-py2.7.egg/Zope2/App/startup.py:344: DeprecationWarning: Expected text
T.note(path)
/home/scmos/scmos.elmirador01/eggs/Zope2-2.13.29-py2.7.egg/Zope2/App/startup.py:357: DeprecationWarning: Expected text
T.setUser(auth_user.getId(), auth_path)
2019-10-22 16:39:12 INFO plone.app.upgrade Dry run selected.
2019-10-22 16:39:12 INFO plone.app.upgrade Starting the migration from version: 5115
2019-10-22 16:39:12 INFO plone.app.upgrade End of upgrade path, main migration has finished.
2019-10-22 16:39:12 ERROR plone.app.upgrade The upgrade path did NOT reach current version.
2019-10-22 16:39:12 ERROR plone.app.upgrade Migration has failed
2019-10-22 16:39:12 INFO plone.app.upgrade Dry run selected, transaction aborted

mauritsvanrees · October 30, 2019, 10:56am

You have selected dry run, so this is just a test, and nothing is saved. Do a normal run, and you should be fine.