Content Import and Export

One idea could be change the Upload to have an advanced tab which allows upload as csv, zip, other site. And hten have a different button for download.

Or rename upload to "File" or "sync" and have upload and download as different tabs in this modal.

1 Like

Yes that could be done but it would be tutorial video. Here I'm trying to ask suggestion for usability features.

@tkimnguyen great idea. But this would be a different project.

Here I'm expecting some usability features and requirements to include in advance tab. Like the ones suggested on the PLIP.

@djay I like the idea but I think first we need to focus on the advance tab of importexport ( accessible from action menu) as the basic tab alone would not meet the admin requirements.

@Shriyanshagro I suspect integrating with the structure pattern in mockup would change your implementation a lot which is why it might worth considering earlier. For a start you get access to resumable uploads, multi-file upload which are part of that pattern. What I'm thinking is the JS could detect if there is a CSV as part of a normal upload and ask fi the user would like to consider it a manifest file with metadata in it. Likewise, a download tab can have a tickbox to either be download as files or files + csv manifest file.

1 Like

please try to keep the code UI agnostic; the main use I see for this new feature is to help on the migration from Plone 4 to Plone 5; if the final result don't solve this use case then it will be not very useful for a bunch of people, including us.

it would be very nice to be able to run a script from the command line and get the content of a site exported.

if that's not possible, it would be very nice to document the expected format so such script can be build outside of this package and we still be able to use it to import content into Plone 5.

@hvelarde plone 4 to plone 5 was not one of the use cases being considered. The current implimentation uses plone.rest which I believe doesn't work for plone 4 so I don't see this usecase working. If what you want is plone 4 to plone 5 then look at jsonify.

The general goal of this project is a nice UX for end users to get content in and out of plone and possibly between plone sites of the same version, not to solve upgrade issues for integrators at this stage.

1 Like

RestApis provide solution for Archetypes as well but I'm not very sure about their limits to this datatype. Thus @hvelarde till certain limits this product can be used for Plone upgradation as well. :slight_smile:

:thumbsup:[quote="djay, post:107, topic:3789"]
What I'm thinking is the JS could detect if there is a CSV as part of a normal upload and ask fi the user would like to consider it a manifest file with metadata in it. Likewise, a download tab can have a tickbox to either be download as files or files + csv manifest file.
[/quote]

Isn't this was already decided that only good_file(zipped BLOB and CSV) would be considered by this product and any other file would be rejected with a log report?
Also uploading only manifest file(with metadata in it) doesn't make sense if export is also not providing the same possibilities

No, there were more use cases than that. This is not just a Plone -> Plone tool.

Ideally we are aiming for all of the following use cases

a) Upload just files/folders. Will create or update existing html and blobs
b) Upload just the manifest. Will update metadata and/or create empty content (depending on settings). This is useful for doing bulk updates of metadata like ttiles, authors, sharing settings, modification dates etc. Manifest might just contain a few columns with different titles than one exported from plone. ie this can be some handmade CSV.
c) Upload files + manifest file. Will create and/or update content including metadata.
d) Upload files + manifest file inside a single zip. Will create and/or update content including metadata.
e) Upload files + manifest file inside a single zip exported from another plone site.
f) Export just files and folders in a zip
g) Export just manifest (perhaps with subset of fields)
h) Export files/folders + manifest in a zip. For example to place in git for backup purposes, or to export content to another system

oh and @tkimnguyen idea
i) upload from another site. Enter url and do auth and stream content directly without download then upload.

Perhaps in terms of UI it could be Actions > Upload and Actions > Download with the same popups integrated into folder contents (ie remove the current folder contents Upload button and replace it with this enhanced upload feature, and add an extra Download button. And also enhance or reuse the existing Upload pattern (which I believe allows Dnd and multifile upload).
I'm just a little concerned as a lot of plugins chuck things under the Actions menu so it can get crowded.

BTW. I think perhaps the terms import/export are what is throwing people off here. This is really about upload/download/update of content and/or metadata. Transporting content between sites or using it for upgrading purposes is a nice side effect of having the ability to download and upload files and metadata together.

Would it be overkill to introduce a new menu entry? I think it is important enough to be a menu by itself. It would avoid the potential clutter in the Actions menu.

We are already trying to reduce the items on the toolbar. There is a very long ticket about it somewhere :slight_smile:
I dont think it will be used that often. Which is why I think under contents is probably enough

Files + manifest file inside a single zip is only considered as a good file. And that fulfils most of these use cases.

Metadata is nedded to map the BLOB to Plone Context

It's already possible. By providing a zip file only with CSV in it.

Files + manifest file inside a single zip is only considered as a good file.

Yes, that can be implemented.

That's currently far out of my understanding to implemen. :sweat_smile: And thus can consider this, only if time permits.

No it's not. If you have folders and files then that is enough. This is how webdav works.
Of course this requires the ability to retain folders during the upload process which has only partial browser support but that will increase over time

I'm saying that it will be more user friendly to not require a zip if you don't want.

Okay, but then the structure of folders has to be specific(like while exporting the data using this product) Because system needs some method to map the file to the object which it belongs to.[quote="djay, post:117, topic:3789"]
I'm saying that it will be more user friendly to not require a zip if you don't want.
[/quote]

Okay, cool that can be done. But in such case again only at max two file can be accepted by the system. One is the CSV file and the other is file structure containing file and folders.
Or else we need to develop a mechanism to map the files to the object.
As Currently the system simply browse through the folders and create a virtual path of each file in memory and on later stage it map those files to the object as CSV file holds path to every files for every object.

The manifest either has a way to map files to their intended destination. ie it has some local_path field + some target_path field. OR the local path is the target path, ie the structure of the folders is how it should be in plone.
The case where there is no manifest file, then that means that the local path is the target path.

Aah, this sounds fine!! I may have to implement some checks to ensure any brokeage in the structure.

New mockups for plone.importexport are here.
Suggestions are always welcome and appreciated. :slight_smile:

Need solution for this . To built a flexible and easy to use UI for this add-on.