Partial streaming of Videos is back

Hustle and Bustle
1

After @mamico wrote https://github.com/plone/plone.namedfile/pull/86 I added one line to wildcard.media (https://github.com/collective/wildcard.media/pull/69) and partial steaming works again.
I uploaded a 2GB video and it worked like a charm :smiley:
Due to https://github.com/plone/plone.formwidget.namedfile/pull/44 the same should work for the default file type if you upload a mp4-video for example.

5 Likes
2

Backport to plone.namedfile for Plone 5.1 seems to work as well on the File Contenttype, but still have to verify a bit more. pull request https://github.com/plone/plone.namedfile/pull/94

1 Like
3

I'm glad to share another tip that I learned using Range support with Varnish.
I hope could be useful to others.
@fredvd could this indication be useful in plone.recipe.varnish?

Varnish manage range headers himself and generally strip the request headers involved on, like Accept-Range, Ranges [1].

I did a quick test:

  1. against plone w/ range support
    % curl 'http://127.0.0.2:8080/Plone2/vid_20141205_231143.mp4/@@download/file/VID_20141205_231143.mp4' -H 'Range: bytes=10-19' -o video.mp4.part
    % ls -l video.mp4.part
    -rw-rw-r-- 1 mauro mauro 10 mag 31 17:29 video.mp4.part
    % tail -1 var/log/instance-Z2.log
    127.0.0.1 - - [31/mag/2020:17:29:42 +0200] "GET /Plone2/vid_20141205_231143.mp4/%40%40download/file/VID_20141205_231143.mp4 HTTP/1.1" 206 10 "-" "curl/7.68.0"
  1. against varnish (default conf) with plone backend
% curl 'http://127.0.0.2:6081/Plone2/vid_20141205_231143.mp4/@@download/file/VID_20141205_231143.mp4' -H 'Range: bytes=10-19' -o video.mp4.part
% ls -l video.mp4.part
-rw-rw-r-- 1 mauro mauro 10 mag 31 17:30 video.mp4.part
% tail -1 var/log/instance-Z2.log
127.0.0.1 - - [31/mag/2020:17:30:55 +0200] "GET /Plone2/vid_20141205_231143.mp4/%40%40download/file/VID_20141205_231143.mp4 HTTP/1.1" 200 19042836 "-" "curl/7.68.0"

Range support went fine for the user, but varnish, removing Range headers, has fetched the complete file from the backend.

  1. against varnish, adding a return(pipe); in configuration, with same plone backend
% curl 'http://127.0.0.2:6081/Plone2/vid_20141205_231143.mp4/@@download/file/VID_20141205_231143.mp4' -H 'Range: bytes=10-19' -o video.mp4.part
% ls -l video.mp4.part
-rw-rw-r-- 1 mauro mauro 10 mag 31 17:32 video.mp4.part
% tail -1 var/log/instance-Z2.log
127.0.0.1 - - [31/mag/2020:17:32:43 +0200] "GET /Plone2/vid_20141205_231143.mp4/%40%40download/file/VID_20141205_231143.mp4 HTTP/1.1" 206 10 "-" "curl/7.68.0"

At the end, in my opinion, using varnish, there are different manner to manage Range requests (from simplest to complex):

  1. with few and small videos (a bunch of MB), leave varnish to manage Range requests could be an option, but, I think, for this case, there is no need to manage Range requests at all.

  2. configure varnish to pipe requests that we need to manage with Range, and handle it with Plone, ie

   sub vcl_recv {
      ...
      if (req.url ~ '\.mp4$') {
          return(pipe);
      }
      ...
   }

  1. leave varnish to manage range request, like 1., but configure varnish with specific storage for large files [2]

  2. Caching partial object in varnish [1, 3] and handle Range support with Plone. In this case, probably, is better also to partition varnish storage [2]

Rif.
[1] https://info.varnish-software.com/blog/caching-partial-objects-varnish
[2] https://info.varnish-software.com/blog/partitioning-your-varnish-cache
[3] https://moz.com/devblog/how-to-cache-http-range-requests

1 Like