Content Import and Export

please try to keep the code UI agnostic; the main use I see for this new feature is to help on the migration from Plone 4 to Plone 5; if the final result don't solve this use case then it will be not very useful for a bunch of people, including us.

it would be very nice to be able to run a script from the command line and get the content of a site exported.

if that's not possible, it would be very nice to document the expected format so such script can be build outside of this package and we still be able to use it to import content into Plone 5.

@hvelarde plone 4 to plone 5 was not one of the use cases being considered. The current implimentation uses plone.rest which I believe doesn't work for plone 4 so I don't see this usecase working. If what you want is plone 4 to plone 5 then look at jsonify.

The general goal of this project is a nice UX for end users to get content in and out of plone and possibly between plone sites of the same version, not to solve upgrade issues for integrators at this stage.

1 Like

RestApis provide solution for Archetypes as well but I'm not very sure about their limits to this datatype. Thus @hvelarde till certain limits this product can be used for Plone upgradation as well. :slight_smile:

:thumbsup:[quote="djay, post:107, topic:3789"]
What I'm thinking is the JS could detect if there is a CSV as part of a normal upload and ask fi the user would like to consider it a manifest file with metadata in it. Likewise, a download tab can have a tickbox to either be download as files or files + csv manifest file.
[/quote]

Isn't this was already decided that only good_file(zipped BLOB and CSV) would be considered by this product and any other file would be rejected with a log report?
Also uploading only manifest file(with metadata in it) doesn't make sense if export is also not providing the same possibilities

No, there were more use cases than that. This is not just a Plone -> Plone tool.

Ideally we are aiming for all of the following use cases

a) Upload just files/folders. Will create or update existing html and blobs
b) Upload just the manifest. Will update metadata and/or create empty content (depending on settings). This is useful for doing bulk updates of metadata like ttiles, authors, sharing settings, modification dates etc. Manifest might just contain a few columns with different titles than one exported from plone. ie this can be some handmade CSV.
c) Upload files + manifest file. Will create and/or update content including metadata.
d) Upload files + manifest file inside a single zip. Will create and/or update content including metadata.
e) Upload files + manifest file inside a single zip exported from another plone site.
f) Export just files and folders in a zip
g) Export just manifest (perhaps with subset of fields)
h) Export files/folders + manifest in a zip. For example to place in git for backup purposes, or to export content to another system

oh and @tkimnguyen idea
i) upload from another site. Enter url and do auth and stream content directly without download then upload.

Perhaps in terms of UI it could be Actions > Upload and Actions > Download with the same popups integrated into folder contents (ie remove the current folder contents Upload button and replace it with this enhanced upload feature, and add an extra Download button. And also enhance or reuse the existing Upload pattern (which I believe allows Dnd and multifile upload).
I'm just a little concerned as a lot of plugins chuck things under the Actions menu so it can get crowded.

BTW. I think perhaps the terms import/export are what is throwing people off here. This is really about upload/download/update of content and/or metadata. Transporting content between sites or using it for upgrading purposes is a nice side effect of having the ability to download and upload files and metadata together.

Would it be overkill to introduce a new menu entry? I think it is important enough to be a menu by itself. It would avoid the potential clutter in the Actions menu.

We are already trying to reduce the items on the toolbar. There is a very long ticket about it somewhere :slight_smile:
I dont think it will be used that often. Which is why I think under contents is probably enough

Files + manifest file inside a single zip is only considered as a good file. And that fulfils most of these use cases.

Metadata is nedded to map the BLOB to Plone Context

It's already possible. By providing a zip file only with CSV in it.

Files + manifest file inside a single zip is only considered as a good file.

Yes, that can be implemented.

That's currently far out of my understanding to implemen. :sweat_smile: And thus can consider this, only if time permits.

No it's not. If you have folders and files then that is enough. This is how webdav works.
Of course this requires the ability to retain folders during the upload process which has only partial browser support but that will increase over time

I'm saying that it will be more user friendly to not require a zip if you don't want.

Okay, but then the structure of folders has to be specific(like while exporting the data using this product) Because system needs some method to map the file to the object which it belongs to.[quote="djay, post:117, topic:3789"]
I'm saying that it will be more user friendly to not require a zip if you don't want.
[/quote]

Okay, cool that can be done. But in such case again only at max two file can be accepted by the system. One is the CSV file and the other is file structure containing file and folders.
Or else we need to develop a mechanism to map the files to the object.
As Currently the system simply browse through the folders and create a virtual path of each file in memory and on later stage it map those files to the object as CSV file holds path to every files for every object.

The manifest either has a way to map files to their intended destination. ie it has some local_path field + some target_path field. OR the local path is the target path, ie the structure of the folders is how it should be in plone.
The case where there is no manifest file, then that means that the local path is the target path.

Aah, this sounds fine!! I may have to implement some checks to ensure any brokeage in the structure.

New mockups for plone.importexport are here.
Suggestions are always welcome and appreciated. :slight_smile:

Need solution for this . To built a flexible and easy to use UI for this add-on.

rest.api issues

Need help with registering JS in the add-on.

Need help on this.

How to handle this exception?

Finally, we’ve reached the end of this summer.

First of all, I would like to thank my mentors @frapell , @djay , @loechel , @cewing and all the community members for their guidance and support through out the period. :innocent:
Here is the final report of my work under Google Summer of Code 2017 programme.

Definitely, I will plan to contribute to Plone in my free time even after GSoC. :slight_smile:

4 Likes