Migrating Content to a newer vanilla version of Plone

Hi Guys - Looking for an inspiration moment to take me to the next step after upgrading to 4.3.9 using MS

At 4.3.6 I had created a content structure (folders) of every airport and city in the world - In the ensuing 20 months many of the addons are no longer required in what I want to do.

Is there a procedural method, (point to a HowTo tutorial and I am happy to teach myself ) to take that 4.3.6 Contents (400+ hours) folder and migrate it to a 4.3.9 vanilla version of Plone.

I have tried various variations by dabbling with backup but am getting differing errors dependent on how I try and do it.

TIA

collective.jsonmigrator

maybe you just can:
uninstall the add-ons you no longer need,
install a fresh 4.3.9
replace the blog and data.fg on the new site with those from the old (basically the /var folder)

Rephrasing both statements above:

Running migration provided by Plone:

  • Self speaking, make a backup.
  • Remove all unnecessary addons.
  • Update Plone to latest 5.0 using buildout (Maybe this is useful: https://github.com/collective/minimalplone5 )
  • Run the migration provided by plone.app.contenttypes. It can also handle custom types to an extend.

Migrating using transmogrifier:

That often involves a lot of work. This is a transmogrifier pipeline, which worked for me in different migration projects: https://github.com/thet/programmatic.transmogrifier
There is also much documentation, unfortunately often outdated. However, for more complex migration projects, this is the way to go.

A few links:

In reply to Espen's suggestion I got the following (this was the easiest to try first)
Will now try AJ's suggestion

Site Error
An error was encountered while publishing this resource.

Sorry, a site error occurred.Traceback (innermost last):
Module ZPublisher.Publish, line 249, in publish_module_standard Module ZPublisher.Publish, line 197, in publish Module zope.event, line 31, in notify Module zope.component.event, line 24, in dispatch Module zope.component._api, line 136, in subscribers Module ZODB.Connection, line 860, in setstate Module ZODB.Connection, line 914, in _setstate Module ZODB.serialize, line 612, in setGhostState Module ZODB.serialize, line 605, in getState Module copy_reg, line 48, in _reconstructorTypeError: ('object.new(ChildSiteDescriptor) is not safe,
use Persistence.Persistent.new()', <function _reconstructor at
0x0000000000A26358>, (<class
'collective.lineage.descriptors.ChildSiteDescriptor'>, <type
'object'>, None))

Troubleshooting Suggestions

The URL may be incorrect.The parameters passed to this resource may be incorrect.A resource that this resource relies on may be
encountering an error.

For more detailed information about the error, please
refer to the error log.

If the error persists please contact the site maintainer.
Thank you for your patience.

Errors of this kind can occur when a class which formerly has not been persistent becomes persistent (maybe also in the inverse direction). In my experience, it is very difficult to migrate existing objects in those cases on the ZODB level.

Hey there. It's not quite clear from your question if you are just after an upgrade from 4.3.6 to 4.3.9, but it seems
that you want to stop using some older products and shift data about. If that is the case, then collective.jsonmigrator and friends (transmogrifier) would be what I use.
I did write an article on this, but it's getting a bit old now:
http://www.jowettenterprises.com/blog/plone-content-migration-using-transmogrifier-and-collective.jsonify

Actually thet's recommendation for https://github.com/thet/programmatic.transmogrifier is of the same family - and so I would include that in my solution, though that's not covered in the article above.

Thank you for all responses - this is my time in this part of Python (I'm a true end-user that dabbles having grown up in MS world)

I am now working on your suggestions using the following start points

  1. collective.jsonify
    https://collectivejsonify.readthedocs.io/en/latest/
  2. collective.jsonmigrator
    https://collectivejsonmigrator.readthedocs.io/en/latest/
  3. collective.transmogrifier

I was hoping to find a simple export/import addon but I am now expanding my learning experience.

BBS with my experiences

Just before you make a lot of work for yourself:

AN error like that has (it looks like) to do with
collective.lineage

Which could mean that your old site had collective.lineage installed (or at least, it had it once).
If you dont need colliective.lineage, be sure to uninstall it on the old site before copying things (and probably pack the database, too)
If you need it on the new site, you could try uninstalling it and reinstalling it in ZMI

I might be completely wrong here, but it should be very fast for you to test, since the 'migration steps you plan to do' are probably a little more difficult if you are ' a little unfamiliar with these things'

By the way:
It can also be possible to

  1. make a new Plone site
  2. copy paste from http://localhost:8080/somesite/manage_main to http://localhost:8080/someothersite/
    in case 'most of the content is OK'. (for example if you installed an add on that only affected a few files (for example videos).
1 Like

+1 for what @espenmn said. going the transmogrifier way without much knowledge about configuring, administering and programming Plone won't get you far.

In your old 4.3 site, upgrade to lineage 1.1.3 and run the upgrade steps provided (via addon control panel or ZMI). Look at the docs at https://pypi.python.org/pypi/collective.lineage/1.1.3 .
For Plone 5 you have to update lineage to latest 2 release. I'd even use the source checkout for the still unreleased 2.1 release.

Have to agree with Johannes, but I'd like to learn as I go. So by using djowett's blog I have achieved Lesson 101. This is my content for Lesson 101 for "What is json in the Plone environment?"

  1. Firstly, what is json. Refer to Wikipedia - https://en.wikipedia.org/wiki/JSON
    (This is essential for people with my level of Python (et al) experience)

1a. To check if you need simplejson, run bin/zopepy and "import simplejson" - if you get an ImportError then you do.

$ bin/zopepy

import simplejson
ImportError: No module named simplejson

  1. Requirement to add the following to buildout.cfg
    2.1. collective.jsonify
    2.2. collective.jsonmigrator
    2.3. collective.transmogrifier
    2.4. simplejson
    and run buildout

  2. After running buildout (with collective.jsonify and simplejson)

  3. Create and add a file (used Notepad++) named json_methods.py in /parts/instance/Extensions/ with the following content (create the Extensions folder if it doesn't exist)
    from collective.jsonify import get_item
    from collective.jsonify import get_children

  4. At this point there is a requirement to goto the ZMI side of Plone
    http://localhost:8080/manage_main

At the right top side you will see a dropdown menu showing [ATContentTypes tool]
Use the dropdown menu and select [Add External Method] and press Add

Add 2 external methods for each of the above get_item and get_children

  1. Now test this by accessing the localhost:8080/get_children and localhost:8080/get_item.

Here I successfully got csv data.

Now to look at Lesson 102 >>

Thx to all who have responded and I will continue to update as I progress.

You should not need to explicitely declare the dependency from simplejson and you probably do not need it.

Until collective.jsonify and collective.jsonmigrator are not released with those two PR merged:

depending on the current pypi eggs is enough to trigger the download of simplejson.

Anyway, you can do everything you want with the json module. The json module is, since ages, in the Python standard library.
The dependency makes sense only for Python 2.4 (Plone < 4).

1 Like

@alert I now have merged what wasn't merged and released both packages:

3 Likes

Many thanks Johannes - as explained I am very much an end-user and will work on what you have given me - but given my lack of experience 'under the hood' with plone I will first have to get around the concept of what 'and setup an import transmogrifier pipeline on the site you’re importing to' means and looks like.

Just learning the rudiments before I start the engine :o)

Michael

@alert good update & thanks!
The thing I love about jsonify / jsonmigrator though is that they can rescue REALLY OLD Plone sites into modern sites. I even ran it on Zope & CMF where the python was so old it didn't have simplejson until we did a wee backport of it :slight_smile: https://github.com/simplejson/simplejson/tree/python2.2