Big files upload error

Plone Support
1

Hi everyone,

This is my first post, so I’ll do my best to explain the issue clearly.

I'm trying to upload a large video file (3.5GB) as a "File" content type in Volto, but I keep encountering an issue. The site itself does not display any errors, but the browser console shows the following message:

Uncaught (in promise) Error: Error reading videotaller.mp4: null  
PromiseFileReader.js:8

I found some guidance in the Discord community and applied the following changes to increase the upload limit:

:one: Zope Configuration Updates

I modified zope.ini and zope.conf as follows:

zope.ini (to increase max request body size)

[server:main]
max_request_body_size = 5368709120  # 5GB in bytes

zope.conf (to adjust form memory limits)

<dos_protection>
    form-memory-limit 5GB
    form-disk-limit 5GB
    form-memfile-limit 4KB
</dos_protection>

:two: Volto Settings

I also checked and attempted to modify maxFileUploadSize in Volto’s index.js, but since its default setting does not enforce a limit, this didn’t resolve the issue.

:three: Current Behavior

  • I can upload smaller files without issues.
  • I don’t know exactly when the error occurs (file size threshold).

Has anyone encountered a similar issue? Is there another setting or configuration I might be missing?

Any help would be greatly appreciated! Thanks in advance. :raised_hands:

1 Like
2

I dont think it is a good idea to add such big files to Plone or any other CMS (so I dont think anyone else 'does that'.
Probably, you want to add and stream it from somewhere else and just link to it it from Plone.

3

We upload files this large regularly to plone. We have several PRs that make this work well that are awaiting review.

4

Thanks for the answer. Since you already have uploaded big files, is there any "quick fix" for this?

5

Do you stream them (.mp4) from Plone ?

6

@santrami do you have nginx or another web server in front of Plone? The limit in nginx: How to edit nginx.conf to increase file size upload - Stack Overflow

1 Like
7

And it is not only nginx. Anything in your stack above plone could have max upload limits. Varnish, traefik, haproxy, nginx as Kim mentioned.

8

We use TUS via volto. This is a PR to achieve that based on the one we use

TUS uploads in chunks so you need to ensure the chunk size is less than the max configured by waitress etc. TUS is really the only sensible way to achieve large uploads because even if you set all your proxy limits to some large amount you will find that some small network connection failure will abort the whole upload and the users will get fustrated.

TUS does require some changes to how you deploy. it either needs a temp folder that all your servers have access to, like EFS or NFS. or it needs a way to ensure the same user always sends to the same instance (session affinity)

There are also some PRs that prevent plone from being very slow at the final part of the upload. Might be just this one faster uploads by avoiding unneed large blob reads by djay · Pull Request #166 · plone/plone.namedfile · GitHub

We use collective.xsendfile for large file downloads and streaming.

3 Likes