I wrote a solution for chunked uploads in Plone 5.2 using the jQuery File Uploader plugin GitHub - imsweb/ims.upload: Plone package for chunked uploads. The primary purpose of this was to (1) support large files and (2) be able to resume partially uploaded files. It does this by creating a content type ChunkedFile that contains Chunk objects, and when the upload is complete these are merged into a regular File.
I'm looking for a new solution in Plone 6 for a few reasons:
Having these as a content type makes the backend manageable but it's clunky. Requires hiding them and setting up extra permissions and cleanup
They are just uploaded with regular XHR requests. I think TUS is more sophisticated by allowed PATCH commands
dropzone.js looks like a better plugin, with better info passed to the server
not easily integrated into folder_contents which has its own upload behavior
In Plone 6, I am considering dropping this entirely in favor of the @@folder_contents upload but I have some questions about it. From what I can tell, it looks like there is a pat-upload pattern in plone/mockup which is built on Dropzone.js, but this fails with a 502 on some files above some size - maybe 1GB. Is this maybe a restriction outside of Plone, such as my Apache webserver? Because from what I can tell in the pat-upload code the max is set to (effectively) unlimited). The progress bar progresses slowly until 100% and then errors out. I don't see anything about setting the chunked flags to true but since I'm seeing this progress, I don't know how it would be able to know this unless it were chunked in some way?
Finally, I looked at the backend @@fileUpload view and noticed that there are some settings for TUS support. These were disabled and my attempts to enable TUS to play with it ended pretty quickly. Installed the tus python package but it errored out because it doesn't seem to support python 3. Not a surprise, given that it looks like it hasn't been updated in 10 years GitHub - vangheem/tus: tus implementation in python
I would appreciate if anyone could clear this up or point me somewhere relevant for insights.
From the way you are writing this it seems like you are referring to plone 6 classic not plone 6 (ie plone 6 volto)
We have some patches to plone 6 to put Tus support back into the UI but haven't got it into a PR form yet. We've had to make changes to the rest end point to get over some speed issues.
As far as I know Tus should still work in classic. No extra imports would be needed.. but I haven't tried it. You should just need to set the env bars on where to put the temp files and set things up to ensure that all instances have access to the same temp file.
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2023.2\plugins\python\helpers\pydev\pydevconsole.py", line 364, in runcode
coro = func()
File "<input>", line 1, in <module>
File "C:\Program Files\JetBrains\PyCharm 2023.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "C:\Users\oolit\PycharmProjects\plone\venv\lib\site-packages\tus\__init__.py", line 4, in <module>
from StringIO import StringIO
File "C:\Program Files\JetBrains\PyCharm 2023.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'StringIO'
That looks like it's not converted from python 2 to me, using StringIO.StringIO instead of io.StringIO. It probably wouldn't be hard to convert it but I do not have the expertise with TUS to debug anything else going on here.
You are right. It does look like the backend for Tus in classic is broken.
There shouldn't be two backends but I'm not sure how hard it is to switch the current upload widget to put in the right auth headers to use the restapi. And I'm not sure restapi is mandatory for classic.
Either way you should create a bug ticket. Seems like there are no tests for TUS support otherwise this wouldn't have gone undetected
Volto doesn't support Tus on the front-end at all so I can't recommend that as a solution... Yet. PR is coming.
