GSOC 2019: plone.importexport

I think doing the plone training.plone.org is a good start.
I think a good start would be looking at some simple bugs in either plone.importexport or in plone itself. We are commissioning some changes to plone.importexport right now but they should be finished in the next day or so so it would be good to see whats left after that is finished.
Plone has many bugs marked as being suitable for beginners. if you look at its github trackers (there are different ones for different modules).

thinking more probably the best way to start is add some tests once the current work on importexport is finished.

2 posts were split to a new topic: Errors installing plone 5 stable

Hello @djay
I'm Aalekh Jain currently pursuing my B.tech and MS from IIIT-H, India.
I'm an Open Source enthusiast having experience in full stack development (primarily in) Python and Javascript and have used tools or frameworks like Django, Nodejs, Celery, Travis, MongoDB, Selenium to name a few, either in my personal projects or as part of our college's annual TechnoCultural festival. I am also an automation lover and love to automate things, be it on software or on a hardware level.

I have been exploring Plone over the past weeks and also played with it on my local machine. I have also gone through the documentation and training as well and I found Plone to be an exciting organization to contribute to.

I've gone through a couple of threads on this topic and have tried to develop a decent understanding on this, which is to develop an addon which would provide an import/export solution even for non-techy users.

Few of the major challenges that are involved which I could recognize are -

  • Providing an easy to use UI which is built on top of the Plone restapi.
  • Handling permissions on the imported/exported content. (Maybe Plone restapi can handle permission as mentioned in their docs, I'm not fully sure here, correct me if I'm wrong)
  • Developing a mechanism to handle existing content and perform required action on them.
  • Handling large files during import/export.
  • Providing a solution for bulk update of required fields/contents.
  • Integrating useful error logs (if any) along with the import/export tasks.

Kindly guide me to proceed in this project for GSOC'19.

Thanks,
Aalekh Jain
https://github.com/ironmaniiith/

1 Like

Welcome to plone @ironmaniiith. All of that sounds good. But perhaps you need to install and use the current version of plone.importexport to get a feel for how it currently works?
You might want to use one of the later branches as some work has already been done on it.
Note however that the UI uses parts of the restapi internally for serialisation but is not actually connecting via REST itself.

2 Likes

A post was split to a new topic: Installing plone.importexport - conflicting dependencies errors

Hi @djay and @Shriyanshagro
I've uploaded my draft on google summer of code website. Kindly review the draft and suggest required changes.

Thanks

I've merged one of the new PRs with lots of changes on and am now testing to see if this helps it build more easily.
Unfortunately the tests still don't pass.

1 Like

This fix could be included as part of GSoC

Definitely, I've included that in my proposal as well.

Hi @djay
The latest commit solved almost all the problems and there were some minor issues which I was able to solve by following some threads on the community.plone. However, for making the project up and running, I had to install some more packages, which I did by

pip install configparser plone.directives.form plone.recipe.zope2instance==4.0

I tried several zop2instance<4 as well, but only 4.0 worked for me

I tried running import.export addon by exporting a zip file and then importing the same, which also generated a log file. Following is the screenshot attached

Sorry for the late response as I was previously involved in drafting the proposal which took me some time.

Kindly suggest how should I further proceed? Meanwhile I am also following training docs.

Thanks

@ironmaniiith Not sure what you mean by log but there is currently a bug that the path exported is not the same kind as is needed for import. There is a PR to fix that which needs to be reviewed and potentially merged - https://github.com/collective/plone.importexport/pull/34. It makes sense that import and export use relative paths.

Also you should be able to work around it by using UID instead.

Hi,
Thanks a lot for accepting my GSOC'19 proposal. :grinning:

2 Likes

Welcome, Aalekh Jain to the Plone community :slight_smile:
Our coding period will start at the end of this month. In the meantime get yourself familiar with the code base and Pycharm. We may have a few standups before the coding period to freeze the specifications.
Have a lively hack this summer!

1 Like

Welcome!

Getting familiar with how tests work in plone would be useful. Fixing the tests in this branch so we can start from a more stable branch would be good.

Since part of the potential work might involve enhancing the UI with javapscript I’d also look at mockup and robot tests in plone so you are familiar with them.

Also have a look through the current issue list but in particular familiarise yourself with

which is the suggested UI to tie many of the features together and I think would be the content of the first part of the project and then looking at handling larger files after that.

3 Likes

Hi @ironmaniiith

I'm your third (backup) mentor for this project. Welcome to the Plone community. Looking forward to working with you.

One of the main reasons why the tests aren't working is that the zip file is that is used in the tests is corrupted.
Zip file:
https://github.com/collective/plone.importexport/blob/master/src/plone/importexport/tests/ImportExportTest.zip

If you try to open the zip file or run the tests, it will throw a corrupted file error. You may need to write a hard-coded data structure as the test data or generate a new zip file to test the importer function.

You can find out more about testing Plone at https://docs.plone.org/develop/testing/unit_testing.html#running-unit-tests

my preference for tests is to avoid hard coded test files if its not too much work to do so. Having a zip in the archive doesn't seem like a great idea as it makes it hard to inspect when it comes time to debug tests. Either have

  1. the test setup create the zip from files in the repo
  2. the test setup create the files from scratch and then zip them
  3. create the content in plone, export it to create a the zip and then use that during import tests.

I'd probably go for 3 but some can argue is not isolating export tests from import tests. But there needs to be lots of tests to ensure the export can always be imported so its perhaps pragmatic.

1 Like

Hi,
I'm trying to play with importexport functionality and everytime I change the python code, I have to kill the server and restart. Is there any way it automatically detects changes or I have to keep restarting the server?

1 Like

In my experience pdb works better for importexport development.
Other addons restart the server on every single change which often breaks the code with dumb errors