How to use transmogrify to import csv files into Dexterity content?

I would like to import csv files into Dexterity content on Plone 5. I've been following the instructions under "transmogrify.dexterity.csvimport" in the documentation for transmogrify.dexterity, but I always end up with a page that tells me "Nothing to Import" after I upload the tarball and import it.

I'm not sure whether I'm missing some steps before uploading the tarball, or if something is wrong with my tarball. I've added collective.transmogrifier, transmogrify.dexterity, and plone.app.transmogrifier to eggs and zcml in the buildout file before trying to upload the tarball. Might anyone have an example tarball that works and that I could try using?

1 Like

for a working example check the CSVSource class in transmogrify.wordpress.

1 Like

I also had an issue using Transmogrify and I looked at the Github repo for an example. Do you think you could explain where else I would have to reference csvsource.py in my Plone instance?

please provide context on what are you trying to accomplish.

I am trying to upload a csv file to my Plone instance, with each row being a new dexterity content type. When I upload the csv file, I want it to create as many "Documents" as I have rows in my csv file and for the data to appear in the "Content" tab on my boilerplate Plone page. I tried to use the documentation on transmogrifier.dexterity.csvimport which is here: transmogrify.dexterity, but that was unsuccessful so I have been unsuccessfully trying to find other ways to do this. Please let me know if I can provide any other relevant information

sorry, but that is not going to work; there are plans to implement content import/export like that but currently that is not supported:

read the documentation on Transmogrifier to understand what are you going to do first, and come back with specific questions if you run into troubles.

Thank you for the response and for linking to the documentation. I have a specific question about the documentation. In the screenshot below, I don't see where the function registerConfig is defined and where to put the xml/html-looking code. The >>> suggests that we should be entering these commands into the plone instance interactive mode. But it's not clear to me where <configure ...> and <transmogrifier ...> go.

Thanks @hvelarde for your responses. It seems like what @peterg17 and I are trying to accomplish (upload a csv file into Plone with each row being a dexterity content item) has been done before, as this blog post explains how to do it. I've actually tried following the instructions in that blog post using a Plone 4 instance instead of Plone 5, but again it resulted in a page saying "Nothing to import." I will look into the Transmogrifier documentation as well, but if there is a way to use transmogrify.dexterity.csvimport instead of starting from scratch, that would be great.

yes, the documentation sucks and that's a common story; sorry about that.

that piece of content must be placed inside a configure.zcml file for Zope to read it and register it. dealing with Transmogrifier is not an easy task if you're a newbie.

ask for help on the IRC channel.

no, that's not the case and you're misunderstanding the content of the blog post; as I said before, you have to understand first how Transmogrifier works.

You can use https://github.com/collective/collective.importexport. It works on plone 5 with dexterity. It will create or update new content. It doesn't handle file data however, but could handle html... I'm not sure. We haven't made a release of it yet but let us know if it breaks

2 Likes

As @hvelarde says you need to have a configure.zcml to enter this into.
Best way to do this is to create a new package under src using mrbob (look up bobtemplates.plone). Then you can add the transmogrifier:registerConfig statement... and the xmlns:transmogrifier line too.
Your package will need to depend on transmogrify.dexterity, collective.transmogrifier & maybe other transmogrifier packages too.
Then you want to make example.cfg a copy of csvimport.cfg in transmogrify.dexterity, and adapt it to your needs.
Adapting it probably only means changing this line

[definitions]
    csv_file = /absolute/path/to/my/data.csv

I think that's a good start, but I've not tried this exactly. So give it a go and feed back here.

Hope that helps

@dianezhou96 I think this should help you too. The main issue with the transmogrify.dexterity example is that importcontext in the above line is just a placeholder for a package name. But if you do things how I've worked out, then you should be able to avoid the tarball stuff mentioned in transmogrify.dexterity docs

1 Like

I should add @dianezhou96 collective.importexport has a nice UI to map the fields of your csv to dexterity fields. It's reasonably easy to use. Just Actions / import-export

it's amazing how many solutions the community has created over the years to solve the same problem… I counted 8!

we need to go ahead with the PLIP, ASAP.

Thank you for your answer! I went through all of the steps you recommended but i'm not sure how to use the package once it's there. Like what commands do I run that actually puts my file called "entries.csv" into dexterity content? How will I know that it worked?

Thanks for the response, could you clarify what you mean by "Actions/ import-export"?

@peterg17 see https://github.com/collective/collective.importexport#import

Hi, sorry for the late reply.

I would either use mr.migrator as mentioned here:
GitHub - collective/mr.migrator: Mr.Migrator: Drive-by transmogrifiction made easy!

OR
collective.jsonmigrator as mentioned in steps 6 - 10 of my old blog post here
http://www.jowettenterprises.com/blog/plone-content-migration-using-transmogrifier-and-collective.jsonify

(you can ignore the rest as you are not needing collective.jsonify)

Both these methods give you a view on the Plone site where you can select the pipeline you have created (example.cfg or "Example pipeline configuration") from a dropdown menu, and run it. If it's not in the dropdown menu, then somehting went wrong in the earlier steps.

I'm having trouble getting collective.importexport to work. It requires collective.z3form.datagridfield > 1.1, but the latest version of collective.z3form.datagridfield that seems to work is 1.1.

@dianezhou96 yes you are right. I fixed some bugs and no one has made a release yet. It is working for me with the latest master (https://github.com/collective/collective.z3cform.datagridfield/commit/dfce6822564e3df02610cd46f66ab961fd5e8df0)

Hi @dianezhou96

You should take a look at plone.restapi You can write a simple Python script converting your CSV to JSON and POST it to your instance. It is a very simple and clean approach we made very good experience with.

Cheers
Tom

1 Like