Migrating Plone 4.2.0.1 site

Hi,
I have been given a legacy Plone 4.2.0.1 (4206) site to migrate from a FreeBSD 9 server that I do not have full access to. I plan to deploy the site on a new Debian VM and migrate to a newer version. I have a full backup of the site directory (data.fs etc.). However, I am unable to install a new instance of Plone 4.2. I have tried to install 4.2 on Debain 8 & 9 and also macOS Catalina and get similar errors on all platforms (see below from Debian 9).
I have successfully installed Plone 4.3 on a new VM, but when the old site filestorage is copied over @@plone-upgrade shows errors resulting from the difference between the 4.2 data and the 4.3 site.
The live 4.2 site @@plone-upgrade page says that 'Your site is up to date', so there does not seem to be a way to upgrade to 4.3 on the live server. Do I need to install an instance of 4.2 locally first in order to start the migration, if so how? I would be grateful if you could advise a viable migration path. Thanks.

Installing Python-2.7.3. This takes a while...
Installing distribute...
/usr/local/Plone/Python-2.7/bin/easy_install missing. Aborting.

Installation has failed.
See the detailed installation log at /home/john/Plone-4.2-UnifiedInstaller/install.log
to determine the cause.
$ tail /home/john/Plone-4.2-UnifiedInstaller/install.log
    self.cmdclass[command] = cmdclass = ep.load()
  File "/home/john/Plone-4.2-UnifiedInstaller/packages/distribute-0.6.24/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/john/Plone-4.2-UnifiedInstaller/packages/distribute-0.6.24/setuptools/command/easy_install.py", line 25, in <module>
    from setuptools.package_index import PackageIndex
  File "/home/john/Plone-4.2-UnifiedInstaller/packages/distribute-0.6.24/setuptools/package_index.py", line 10, in <module>
    from md5 import md5
  File "/usr/local/Plone/Python-2.7/lib/python2.7/md5.py", line 10, in <module>
    from hashlib import md5
ImportError: cannot import name md5

That is strange: hashlib should have md5.

Are you sure you have the system requirements fullfilled? Plone typically requires that for some (system related) libraries the development (and not only the production) packages are installed.

Thanks. I think the issue might be the Debian system python 2.7 not being built with SSL support. Looking at the system requirements I need the following:
Python 2.7 (dev), built with support for expat (xml.parsers.expat), zlib and ssl
What is the best way to build this?

One way is to build a local python - as follows:

  • fetch the Python source code and unpack
  • run ./configure
    use ./configure --help to learn which options are supported and use the options adapted to your requirements
  • run make; make tests; make [alt]install

The make builds all extensions whose dependencies (on system libraries and packages) are fulfilled. Some extensions depend on "*-dev" packages. To learn about the dependencies of a specific extension, look into Modules/Setup. This tells you which (system) libraries and corresponding includes are required. You must use utilities of your package management system (e.g. apt-file) to find out which system package contains the required resources.

1 Like

Thanks so much, this solved the issue! :grinning:

There seems to be an issue specifically with the 4.2 UnifiedInstaller and Debian's Python 2.7. It seemed to be enough just to configure the Python location and then make. In case of future reference, these were the steps I followed:

1) Get Python 2.7.18

sudo wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz
sudo tar -xf Python-2.7.18.tgz
cd Python-2.7.18/
./configure --help

2) Install dependencies

sudo apt-get install python-setuptools python-dev build-essential libssl-dev libxml2-dev libxslt1-dev libbz2-dev libjpeg62-turbo-dev libz-dev
sudo apt-get install libreadline-dev wv poppler-utils

3) Run Python configure and make

./configure --prefix=/usr/local/Plone
sudo make && sudo make install

4) Download and Install Plone 4.2

wget --no-check-certificate https://launchpad.net/plone/4.2/4.2/+download/Plone-4.2-UnifiedInstaller-update-1.tgz
tar -xf Plone-4.2-UnifiedInstaller-update-1.tgz
cd Plone-4.2-UnifiedInstaller
sudo ./install.sh standalone --password=mypassword --target=/usr/local/Plone --with-python=/usr/local/Plone/bin/python2.7

5) Buildout and run Plone

cd /usr/local/Plone/zinstance/
sudo bin/buildout
sudo -u plone /usr/local/Plone/zinstance/bin/plonectl start
1 Like

I am now the stage where I need to upgrade the site from 4.2.0.1 to 4.3-latest. I have followed the Upgrading Plone 4 within 4.x.x series dot minor releases instructions and added http://dist.plone.org/release/4.3-latest/versions.cfg to extends in buildout.cfg. However, I get an error because the version of buildout is not a high enough version:

sudo -u plone bin/buildout -c buildout.cfg
While:
  Installing.
  Getting section instance.
  Initializing section instance.
  Installing recipe plone.recipe.zope2instance.
Error: There is a version conflict.
We already have: zc.buildout 1.4.4
but zc.recipe.egg 1.3.2 requires 'zc.buildout>=1.5.0'.

If I run less bin/buildout I can see that 1.4.4 is still being used, even if I use virtualenv and upgrade buildout via pip:

#!/usr/local/Plone/Python-2.7/bin/python

import sys
sys.path[0:0] = [
    '/usr/local/Plone/buildout-cache/eggs/distribute-0.6.27-py2.7.egg',
    '/usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg',
  ]

import zc.buildout.buildout

How can I force buildout to upgrade or stop using the version in /usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg? Thanks.

I see two potential problems:

  1. the new versions should replace the old versions; i.e. you should not add the 4.3 versions to extends but replace an entry there. Likely, this is not your primary problem
  2. zc.buildout is managed differently from others eggs (because it is required for buildout itself). While other eggs live in the "buildout eggs" directory, zc.buildout lives in the Python virtualenv itself. Thus, first upgrade zc.buildout there to the required version. Then use the buildout binary from this virtualenv to run buildout bootstrap in your "instance directory". This will recreate bin/buildout and lets it use the right version. Note that distribute/setuptools, too, is managed like zc.buildout and may also need to be upgraded in the virtualenv.

Note that the Python installation used by your old instance could be too old. Maybe, you use the occasion and upgrade it as well.

1 Like

Usually you would create a fresh virtual environment and install the exact versions for setuptools and zc.buildout as they are specified within the versions.cfg of Plone.That's why usually maintain a bootstrap.sh script performing a pip install with the exact versions needed for a particular Plone version. Yes, it is a notorious pain in the brain.

1 Like

This is the bootstrap.sh I use to pip install setuptools and buildout to the exact versions to what a particular buildout cfg needs.

[ ! -f bin/pip ] && virtualenv .
bin/pip install --upgrade pip setuptools zc.buildout
bin/buildout -c $1 annotate | tee annotate.txt | grep -E 'setuptools *= *[0-9][^ ]*|zc.buildout *= *[0-9][^ ]*'| sed 's/= /==/' > requirements.txt
cat annotate.txt
cat requirements.txt
bin/pip install --upgrade -r requirements.txt
1 Like

Thanks for all the help so far. Given the deadline that I have for shutting down the old server, I now plan to set up a new 4.2.0.1 site for now and focus on upgrading later.
I have 4.2.0.1 running on a new Ubuntu 20 LTS server, but I am getting errors on some pages. It might be connected to images, the problematic pages seem to use @@image resizing. I have tested PIL via the CLI and is working fine. The dependencies are all installed. Traceback:

Traceback (innermost last):
  Module ZPublisher.Publish, line 126, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 46, in call_object
  Module Shared.DC.Scripts.Bindings, line 322, in __call__
  Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
  Module Products.CMFCore.FSPageTemplate, line 237, in _exec
  Module Products.CMFCore.FSPageTemplate, line 177, in pt_render
  Module Products.PageTemplates.PageTemplate, line 79, in pt_render
  Module zope.pagetemplate.pagetemplate, line 113, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 888, in do_useMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 954, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 858, in do_defineMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 852, in do_condition
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 742, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 218, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: file:/usr/local/Plone/buildout-cache/eggs/plonetheme.sunburst-1.2.5-py2.7.egg/plonetheme/sunburst/skins/sunburst_templates/main_template.pt
   - Line 76, Column 20
   - Expression: <StringExpr u'plone.contentviews'>
   - Names:
      {'container': <PloneSite at /mursi>,
       'context': <ATDocument at /mursi/neighbours/index_html>,
       'default': <object object at 0x7fce805f5bf0>,
       'here': <ATDocument at /mursi/neighbours/index_html>,
       'loop': {},
       'nothing': None,
       'options': {'args': ()},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0x7fce773442b8>,
       'request': <HTTPRequest, URL=http://example.com:8080/mursi/neighbours/index_html/document_view>,
       'root': <Application at >,
       'template': <FSPageTemplate at /mursi/document_view used for /mursi/neighbours/index_html>,
       'traverse_subpath': [],
       'user': <PloneUser 'john'>}
  Module zope.contentprovider.tales, line 80, in __call__
  Module plone.app.viewletmanager.manager, line 154, in render
  Module plone.app.viewletmanager.manager, line 85, in render
  Module plone.app.layout.viewlets.common, line 48, in render
  Module Products.Five.browser.pagetemplatefile, line 125, in __call__
  Module Products.Five.browser.pagetemplatefile, line 59, in __call__
  Module zope.pagetemplate.pagetemplate, line 113, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 852, in do_condition
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 531, in do_optTag_tal
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 583, in do_setLocal_tal
  Module zope.tales.tales, line 696, in evaluate
   - URL: /usr/local/Plone/buildout-cache/eggs/plone.app.layout-2.2.6-py2.7.egg/plone/app/layout/viewlets/contentviews.pt
   - Line 6, Column 4
   - Expression: <PathExpr standard:u'view/prepareObjectTabs'>
   - Names:
      {'args': (),
       'container': <ATDocument at /mursi/neighbours/index_html>,
       'context': <ATDocument at /mursi/neighbours/index_html>,
       'default': <object object at 0x7fce805f5bf0>,
       'here': <ATDocument at /mursi/neighbours/index_html>,
       'loop': {},
       'nothing': None,
       'options': {},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0x7fce7735b940>,
       'request': <HTTPRequest, URL=http://example.com:8080/mursi/neighbours/index_html/document_view>,
       'root': <Application at >,
       'template': <Products.Five.browser.pagetemplatefile.ViewPageTemplateFile object at 0x7fce79d3b690>,
       'traverse_subpath': [],
       'user': <PloneUser 'john'>,
       'view': <Products.Five.viewlet.metaconfigure.ContentViewsViewlet object at 0x7fce77348650>,
       'views': <Products.Five.browser.pagetemplatefile.ViewMapper object at 0x7fce773482d0>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 155, in _eval
  Module Products.PageTemplates.Expressions, line 117, in render
  Module plone.memoize.view, line 47, in memogetter
  Module plone.app.layout.viewlets.common, line 253, in prepareObjectTabs
  Module plone.memoize.view, line 47, in memogetter
  Module plone.app.layout.globals.context, line 237, in actions
  Module Products.CMFPlone.ActionsTool, line 80, in listActionInfos
  Module Products.CMFCore.ActionInformation, line 206, in __getitem__
  Module Products.CMFCore.Expression, line 47, in __call__
  Module Products.PageTemplates.ZRPythonExpr, line 48, in __call__
   - __traceback_info__: portal.portal_redirection.isRedirectionAllowedFor(object)
  Module PythonExpr, line 1, in <expression>
AttributeError: isRedirectionAllowedFor

The problem you see comes from an "action" (likely managed via portal_actions or portal_types (if it should be a so called "object action")). This action uses an expression (maybe as "condition") which raises the exception. You can work around the problem by locating and changing this action.

Newer Plone versions (I looked at Plone 5.2) no longer have portal_redirection. Thus, in case, you plan to upgrade in the future, you will need to adapt the action anyway.

1 Like

Thanks, I have now fixed this issue. A too recent version of redirection tool was being picked. I have now pinned the version to the correct one for Plone 4.2, but I now get a different error:
AttributeError: 'NoneType' object has no attribute 'absolute_url'

Traceback (innermost last):
  Module ZPublisher.Publish, line 126, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 46, in call_object
  Module Shared.DC.Scripts.Bindings, line 322, in __call__
  Module Products.PloneHotfix20121106.python_scripts, line 63, in _patched_bindAndExec
  Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
  Module Products.CMFCore.FSPageTemplate, line 237, in _exec
  Module Products.CMFCore.FSPageTemplate, line 177, in pt_render
  Module Products.PageTemplates.PageTemplate, line 79, in pt_render
  Module zope.pagetemplate.pagetemplate, line 113, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 888, in do_useMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 954, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 858, in do_defineMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 954, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 954, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 946, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 858, in do_defineMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 888, in do_useMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 533, in do_optTag_tal
  Module zope.tal.talinterpreter, line 518, in do_optTag
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 858, in do_defineMacro
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 954, in do_defineSlot
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 852, in do_condition
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 531, in do_optTag_tal
  Module zope.tal.talinterpreter, line 513, in no_tag
  Module zope.tal.talinterpreter, line 343, in interpret
  Module zope.tal.talinterpreter, line 742, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 218, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: file:/usr/local/Plone/buildout-cache/eggs/Products.Archetypes-1.8-py2.7.egg/Products/Archetypes/skins/archetypes/widgets/rich.pt
   - Line 21, Column 12
   - Expression: <PathExpr standard:u'accessor'>
   - Names:
  {'container': <PloneSite at /mursi>,
   'context': <ATDocument at /mursi/introducing-the-mursi/environment/index_html>,
   'default': <object object at 0x7f2a5f473bb0>,
   'here': <ATDocument at /mursi/introducing-the-mursi/environment/index_html>,
   'loop': {},
   'nothing': None,
   'options': {'args': ()},
   'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0x7f2a5506d2b8>,
   'request': <HTTPRequest, URL=http://example.com:8080/mursi/introducing-the-mursi/environment/index_html/document_view>,
   'root': <Application at >,
   'template': <FSPageTemplate at /mursi/document_view used for /mursi/introducing-the-mursi/environment/index_html>,
   'traverse_subpath': [],
   'user': <User 'admin'>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 155, in _eval
  Module Products.PageTemplates.Expressions, line 117, in render
  Module Products.Archetypes.ClassGen, line 56, in generatedAccessor
  Module wicked.fieldevent, line 29, in render
  Module zope.event, line 31, in notify
  Module zope.component.event, line 24, in dispatch
  Module zope.component._api, line 136, in subscribers
  Module zope.component.registry, line 321, in subscribers
  Module zope.interface.adapter, line 585, in subscribers
  Module wicked.fieldevent, line 16, in notifyFieldEvent
  Module zope.component._api, line 107, in getMultiAdapter
  Module zope.component._api, line 120, in queryMultiAdapter
  Module zope.component.registry, line 238, in queryMultiAdapter
  Module zope.interface.adapter, line 532, in queryMultiAdapter
  Module wicked.fieldevent.meta, line 78, in field_value
  Module Products.Archetypes.Field, line 1411, in get
  Module Products.Archetypes.BaseUnit, line 103, in transform
  Module Products.PortalTransforms.TransformEngine, line 173, in convertTo
  Module Products.PortalTransforms.chain, line 54, in convert
  Module Products.PortalTransforms.Transform, line 198, in convert
  Module plone.outputfilters.transforms.html_to_plone_outputfilters_html, line 47, in convert
  Module plone.outputfilters, line 6, in apply_filters
  Module plone.outputfilters.filters.resolveuid_and_caption, line 102, in __call__
  Module sgmllib, line 104, in feed
  Module sgmllib, line 138, in goahead
  Module sgmllib, line 296, in parse_starttag
  Module sgmllib, line 338, in finish_starttag
  Module plone.outputfilters.filters.resolveuid_and_caption, line 326, in unknown_starttag
  Module plone.outputfilters.filters.resolveuid_and_caption, line 241, in resolve_image
AttributeError: 'NoneType' object has no attribute 'absolute_url'