How does one use collective.transmogrifier?

So I am currently working on a Plone project which is going to require transferring information into plone using transmogrified. However, I looked through the documentation of transmogrifier than there just seems to be so much missing. For example, I downloaded transmogrified using buildout but I have no idea how to use it to transfer information. Is there can UI somewhere where I can drag and drop files into so that can be converted into plone?

Thank you for your time.

1 Like

I don't think there's a UI for it but I know that the team working on Plone.org has a good set of transmogrifier pipelines they've created to migrate plone.org content to the new site.

Could someone from that team jump in here? @ramon @sneridagh @robzonenet

1 Like

You might also want to ask in the IRC #plone chat room

transmogrifier is powerful but heavy. No UI. Everything needs to be thought out and added to a package.

Take a look at: http://docs.plone.org/develop/plone/content/importexport.html#transmogrify-dexterity-json-import-export

Also, take a look at: https://github.com/collective/collective.jsonify

2 Likes

Good docs & tutorials are great for this. I've been meaning to set it up since the university had so many sites that needed upgrading. A GUI would be great :wink:

I've done a Plone 2.1 to Plone 5 and Plone 4.3 (Archetypes) to Plone 4.3 (Dexterity) migration using this toolset:

collective.jsonify
collective.jsonmigrator
collective.transmogrifier
mr.migrator
plone.app.transmogrifier
transmogrify.dexterity

It took me countless hours to get everything right. But now it works fine and new migrations are done much qicker. I planned to write an article but some project related time constraints prevented me to do so.

Regarding the GUI: mr.migrator lets you start the migration and do some config settings from Plone itself (via the @@mr.migrator view).

1 Like

So I made am imgur folder with pictures of all of the steps that I am taking. The only difference is that normally I use a csv file within my tarball which looks like this:

_path,_type,title,description,type_id
/mytest,testtype,"MyTest","",5

The imgur folder

Can anyone see anything wrong with what I am currently doing? If people have sample tarballs that they would like me to test for the sake of debugging I would be willing to try it.
I need to be able to use transmogrify for research and I can't not use it.
Thank you for your time.

Hi,
I wrote a blog article on using jsonmigrator & collective.jsonify, but I think that's off subject here.

I recommend you looking at Jamie's blog on Importing Dexterity content via CSV

Hope that helps!

1 Like

Can someone give me a tarball that they happen to know works? I would like something that I can debug with.

Take a look at https://github.com/plone/ploneorg.core which leads to https://github.com/collective/ploneorg.migration. Not sure what you mean by a tarball.

This is not what I am looking for. I am attempting to migrate CSV files into plone, not migrate different instances of plone to plone.

Have you tried something like this? https://gist.github.com/mpampols/2256740 which I found via https://duckduckgo.com/?q=plone+import+csv Why do you need to use transmogrifier to do that? It would help if you described your requirements a bit more.

So I talked to someone and I was able to get this to work on a droplet plone instance but I am not able to get this to work on my own personal plone instance. Does anyone have any ideas?

Thank you

I want to migrate the conten of a PLONE 3 Site to a PLONE 5. Ihave installed all above mentioned Products on the target site and collective.jsonify on the PLONE3 source site.
When I start the migration through the @@mr.migrator view with the catalog query option it gets all the items, but except the folders they are all empty (news items, pdf files, image, event with no content).
Have you an idea, what I do wrong?

Hello,

I had the same problem for migrating the news of a Plone 3 site.
I did a two steps migration: 3->4 and 4->5.

Plone 3.2.2 (collective.jsonify) -> Plone 4.3.1 (collective.jsonmigrator)

Then, I installed Plone 5.0.6 and did a
rsync of var/filestorage and var/blobstorage to the Plone 5.0.6 instance

Then I ran the Plone 5 instance, migrated the site, converted to Dexterity
and exported (zexp) the news directory to the new site.

For me, it worked fine (after long hours of trials and errors).

Regards

1 Like

https://docs.plone.org/manage/upgrading/intro.html#no-large-leaps

First, don't do a 3 > 5 migration. Follow @tkimnguyen advice, https://docs.plone.org/manage/upgrading/intro.html#no-large-leaps.

Second, unfortunately, there's no step by step documentation on how to migrate Plone Sites from older versions to another that would work in all cases using transmogrifier. It's a trial and error approach.

You can try to read some code searching for collective.jsonify and other packages named by @thet in github, and try to find projects that uses them as dependencies to migrate to understand how they do it.

Thanks a lot for your advices.

I tried it now in two steps:

Plone 4 to Plone 5 in the way as Norbert descried. This worked fine, including AT to Dexterity conversion.

From Plone 3 to Plone 4 I used collective.jsonmigrator (having collective.jsonify in the source site).
It worked now better than my try from Plone 3 to Plone 5 (as described in my last post): It did get the folders, news items and events with all their contents correctly to the target site.
But from the pdf files and images just the catalog entries were created, but the files itself were still missing.

Besides the pipelines that come with collective.transmogrifier I tried pipelines with different sections as descibed in these foils of Daniel Jowett: https://www.slideshare.net/danieljowett1/json-migrator-41253629. But either they created errors or they did not change the results.

So it looks as if everything comes over, but not items that had to go into the blob storage - which did not exist in Plone 3.

Maybe you can show me the pipelines/blueprints that you used. Thanks.

I seem to recall there being a special way of handling upgrades of pre-blob storage sites.

You probably have to set the JSONIFY_MAX_FILESIZE in your source instance high enough to expose the binary data within the json representation.

environment-vars =
    JSONIFY_MAX_FILESIZE 200000000
1 Like