GSoC 2018 Ideas: Improve Plone's import/export story

@kakshay21 the heart of the problem is to improve the UI to make it easy. This is not a technical issue of supporting more formats. That just increases the maintenance burden for little gain. I don't know who suggest XML but I can't see it adding anything over JSON. Please exclude it.

1 Like

I'm +1 on leaving out XML. It's overcomplicating things.

1 Like

@kakshay21Plone is about empowering normal users, so User Interfaces and Usability is much more important than anything else. @djay recommended CSV last year as the format, as almost every user could work with that, even if it has technical problems.

I guess we have here an old pillow fight about how to use current technologies to gain advantages out of them.
Me personally I would always prefer XML over JSON or other data formats. But it is not about data formats.

As @zopyx pointed out, plone.restapi is the way to go, his assumption is wrong. Plone.restapi is fantastic if you have externals scripts that do everything for you, but plone.importexport is about a Plone Add-On that lifes in Plone and provides the normal users an Intereface to export certain parts or the whole Plone Site to be reimported in other instances. So it is much more about Usability than calling restapi functions.

For me the current state of that add-on is far from being perfect. User Interfaces are a hard job, and they will need several iterations of optimization.

What I do see about the data formats, is, that we currently focus on Plone to Plone migration, but not the orgininal intend of allowing the Users an external modification and processing of the data, before reimport them. For me there is the importance to know the content type schema of each class or object.

Alexander

1 Like

Seems like "small pillow fight" is getting bigger. Well, It always happens in Open Source Community.
But I'm bounded by time to upload my final proposal to GSoC. And If we don't finalize this by today then let's just say "CSV and JSON", and we'll finalize it over the community bonding period with more people to present their views and more arguments😅 and more collaborations??
@loechel @djay @tkimnguyen @zopyx

2 Likes

I am talking of the JSON data format here...no need to reinvent a new JSON format here,

"Just make it better". If we all agree that's what you'll do, that's good enough for me :slight_smile:

2 Likes

Often in open sores (I'm sure Plone is not special in this way) you just have to say "phuck it I'm going for it" after listening to endless debates. At the end of the day, code talks. As Nathan's t-shirt says, "More code, less talk" :smiley:

2 Likes

...and by G*d that is what makes America great: everyone is willing and able (and maybe very ready) to say "f you and get the f outa my way" :slight_smile:

/me feeling twitchy after figuring out my !@#$!@#$ buildout problem

...and you'll wrest the 2nd amendment out of my cold dead hands

because Amerika is so great :yum:

1 Like

These two points I completely agree with. That is what the focus of the project should be on. Making a importexport system that handles many more usecases non technical users might have. Weather they an excel file with dates and titles, or a zip of images, or they require a report on some custom fields for a custom content type, or they want all their blog posts exported or they want to export everything just in case they need it later, or they want some data to hand to a developer to import into wordpress, or they are given some content and a csv that has to be imported, etc, etc etc.

XML is nice to have in addition to JSON but mostly doesn't help with any of the above. Any XML format has to be transformed into something else to be useful for some other system so a developer is needed and then if they use json or XML, its probably not that hard either way.

CSV/excel and zips of files is what is most useful if the user isn't a developer.

1 Like

@loechel here are you saying that the exported content should include the content type definitions along with the data? So similar to the original proposal of somehow combining in what collective.themesitesetup or genericxml/portal_setup does into a single file?

1 Like

XML is likely the best format when it comes to well-defined data-structures, validation and transformation but it's the wrong format for such a purpose here. Basically all solutions in the past for serializing and deserializing Plone content failed in some way or were not suitable in production.

CSV is a pretty stupid format and you have to deal with all problems regarding quoting, encoding, separator etc. - besser use Excel format if you want to support a spreadsheet format or ODF.

JSON is likely what you want with all its disadvantages (limited types (e.g. no support for datetime)), poor support for defining schemas (JSON schema -> :put_litter_in_its_place:).

if you have the export-import chain directly in your hands using well-defined add-ons then everything is fine.
The problems arise if you get data from non-Plone sources. You need a strong validation on the import side here and deal with all aspects of cross-reference, datetime related aspects (timezones)...

Hi @encolpe, @cewing i am Mua Laurent final year student from the univerisity of Buea, Cameroon and an open source enthusiast. Joined open source last year and i am very much loving it here my github profile - https://github.com/muarachmann. I would love to work on this project very much for this upcoming GSOC section. I have read through the discussion and it is very very interesting and important feature needed in plone. I have skill set in mysql, mariadb, python, php, javascript , HTML and css. while python is my most preferred language i also know some C and Java language with little knowledge of golang, i would love to use this as an opportunity to better my skills in python and also participate in the growing of the community. I have already cloned Plone. Please can you tell me the way forward i would be glad to get started by first solving issues around and get familiar to the code base. Looking forward to hearing from you thank. :slight_smile:

Hi. It's great you're interested. You might want to look at the proposal for this years GSOC importexport project rather than last year and see if it interests you. https://community.plone.org/t/gsoc-2019-plone-importexport/

Thanks @djay i will get to it