Introduction and queries regrading 'TUS large uploads' Project for GSoC 2024

1

Hello everyone!

My name is Ravjot Singh, currently a second year student at Indian Institute of Information Technology and Management, Gwalior and I'm deeply passionate about the world of full stack web development. I'm proficient in JavaScript, Typescript, C/C++, Python, Java, React, Redux, Node.js, Express, MongoDB, MySQL, PostgreSQL and have also worked with CI/CD Jenkins, Docker, AWS.

Ever since the "Allow reliable large uploads in Volto using TUS" project got announced back in February, it really caught my eye. I haven't gotten around to wrapping up a proposal just yet, but I've been diving into all the materials I can find to really get a handle on what the project needs and how we might tackle it.

Here are the resources I've explored:

My understanding about this project

  • Using tus-js-client for handling TUS (resumable file uploads) due to its lightweight (approximately 60kb minified) and Semantic UI for the frontend design.
  • By selecting "Add > Upload files," users would be taken to a bulk upload page, where they can upload multiple files at once.
  • Using ZODB (Zope Object Database) blobs to store partial uploads, with the idea of joining these partial uploads once the entire file has been transmitted.

I'm piecing together my GSoC proposal on "Implementing TUS in Volto" and there are a couple of points I’d love to clarify to ensure I’m on the right track:

  • For the integration of TUS, would it be preferable to establish a dedicated upload page, akin to react-uploady's approach? This could replace the traditional file view, streamlining the upload process.
  • TUS protocol supports chunked uploads, but what’s our plan for chunk size customization? Should it be a fixed size, or do we provide the flexibility for users to set it based on their network stability?
  • I'm inclined to propose that the upload action is triggered upon saving to provide editors with the flexibility to review before committing to the upload. We could mitigate any potential user confusion with a clear UI prompt indicating that saving is necessary to complete the upload process.

I'm looking forward to contributing to this project and feel that getting these questions answered will help me put together a solid proposal for you to consider. I would greatly appreciate advice from any mentor or insights from fellow candidates. I would really appreciate any little useful insight from the mentors for this project @djay @JeffersonBledsoe

Thanks so much for your guidance and support!

Best,

Ravjot Singh

2

As the PLIP mentions there are many places the UI needs to be changed.

  1. On the contents view there is an upload button that opens a dedicated upload view.
  2. this same upload view will be made accessible from the add menu
  3. in the sidebar where there is a file field (for blocks or for content type metadata) then this file widget will need to change to display progress and use TUS.
  4. some blocks like the image block perform an upload so these will need to change also.

This will be controlled by the configuration during deployment via an environment variable. In our implementation 50Mb seems to work well for most cases. it's more important that the ops have control over this due to potential body size limitations in reverse proxies that might be being used. If there is a need for end user adjustment it can added as a feature later.

Even if the choice is to upload the files before saving the blobs are not committed in a transaction so it won't change how the end user should work.
The choice is more likely determined how easy it is to integrate into the volto framework and how reliable it will be in practice.

1 Like
3

I'am Ravjot Singh, I've crafted a proposal for the "Allow Reliable Large Uploads in Volto Using TUS" project. I've submitted it to the Google Summer of Code website. I'm eager to hear your feedback on my plan. I look forward to working under your guidance and hope to learn and contribute effectively to the project. :grinning: :grinning:

4

@djay I was trying to figure out integration of Tus endpoints into my projects, i have tried many things its giving cors error every-time. I have gone through

Can you suggest some reference project or documentation from where i can go through its implementation.