What is wrong with plone.scale?

Development
1

Plone image scaling support is causing performance issues with headless use-case and Volto. Something should be done before Plone 6 or soon after.

I heard that at Alpine City Sprint 2020 there was discussions about completely removing image scaling and replacing it with X. Who could provide more details? @jensens?

I'm aware of some issues (too eager cache purging, ineffective caching keys, not recognizing SVGs), but not all. How broken it really is?

Cancellation of PLOG gives me a few days to work on something that would benefit Volto and GatsbyJS use-cases. Because I already have tweaked plone.scale for own use cases, I'd be interested in one more try to save it.

@fredvd I heard that you looked image scales in depth I'd love to hear your insights about everything that is wrong / broken in it currently.

2 Likes
2

Some ideas of my own:

  • Instead of write on read, we should update all "known scales" immediately when image is updated. Write is "slow" anyways, and when doing multiple scales at once, it is possible to generate those in parallel with multiprocessing.

  • plone.scale cache keys should be deterministic, derived from bytes and scaling configuration; currently they are just random UUIDs

  • scales should not auto-expire as they do now

3 Likes
3

From what I heard talking to people this is an issue with volto/restapi and their need to create all the image scales at once instead of doing that lazily.

Probably creating the scale is necessary to have the URL to fetch the scale.
If this is the case the process should be decoupled.
BTW do we have a github issue about this?

2 Likes
4

Already with deterministic UUIDs it would be possible to return the URLs in REST API already before the scales are generated, and continue generating them with write on read.

But probably avoiding write on read and generating all scales in parallel when image is updated would be the best. :thinking:

2 Likes
5

Thanks for picking this up Asko! So here is our use-case/problem:

Volto allows modern, image-heavy, grid-based composite pages (e.g. the front page) where we have to serve 30-40 large images per page. If you have 20-100 of those pages (which usually are the most visited ones), and you generate image scales on the fly, like plone.scaling does, you have a scalability problem on a high-profile website with lots of visitors.

What we did so far to get away with this is:

  • Make sure we use Plone image scales (the Volto flexibility of our grid-blocks make this hard because an image in a grid block does not know what kind of scale it needs, it could span over the full width or only take 1/4 of full width)
  • Aggressively cache all image resources
  • Use a custom warm-up script that crawls those composite pages and warms up the cache
  • Fix performance bottlenecks in Zope/Plone (thanks @jensens! :))

Though, we have to address some fundamental design problems IMHO that makes the Plone backend ready for the new decoupled frontend use cases. Plone was never meant to be used in this way on a high scale. Guillotina was written from scratch with that use case in mind. We have to see how far we can push Plone. :slight_smile:

We have two possible ways to solve this problem:

a) try to improve Plone itself

b) lift-off the problem/work to a separate image server (like Thumbor)

I am totally fine to explore option a) at first since something like Thumbor will always be an add-on for larger sites (similar to Solr or ES for search). So improving the image scaling for Volto and for existing Plone sites is something I'd be more than happy to help with.

4 Likes
6

@datakurre Hi Asko. Yes, I 'know' a bit more. I met Markus (@iham) at the AlpineCitySprint in february where he proposed a sprint topic on image handling in Plone. We discussed a lot of issues, wishes, features with the others participants in Innsbruck @jensens was also involved and I'm forgetting others. We got quite a bit of 'different' features in different area's, itches you don't have in your own projects, but are very valid.

My 'personal' itch to look at image handling (again) is that we have a customer site with 5000 images that have to be auto-focal-point'ed. @mauritsvanrees and I looked at it and the only solution so far is to overwrite @@images and/or create our own @@customimages scaler. One of my idea's was to create a 'dynamic' scale, where I could put "@@images/image_field/autoscale_600x400" in a template, a bit like the placeholder services you find online. (I'll get back to this).

We also discussed thumbor at the alpinecitysprint (GitHub - thumbor/thumbor: thumbor is an open-source photo thumbnail service by globo.com). It is a separate image scaling/serving/caching/modification service. It has been developed as the 'external' image inserting service for message boards (globo.com)

Thumbor looks very interesting, but the project was still stuck on Python 2.7 in February and the tickets on github for migration to python 3 seemed stalled. An alternative/replacement mentionned by the Thumbor people in their issues is to switch to Imageflow (GitHub - imazen/imageflow: High-performance image manipulation for web servers. Includes imageflow_server, imageflow_tool, and libimageflow) but this project doesn's seem to be 1.0ish yet and mainly a library.

I attended the PloneTagung in Dresden in March 9-13th, both Timo and I suggested an Open Space on image handling. We did this on wednesday and it was very well attended. We received nice feedback from developers from the Neos CMS who are already using a 'decoupled' image handling service.

And at the Open Space we found out Thumbor's Python 3 initiative was not dead after all. They released 4 or 5 Python3 alpha's since February :slight_smile:

After a general introduction the Open Space focussed a lot on how Thumbor works @jensens gave a good explanation that you have to be very careful with opening up to DDOS-attacks on the image server if you allow external sources to just request any kind of transformation on an image (and that a milion times). The CMS 'signs' a list of non destructive edits, delivers this to the client and the client can use this signed ticket to request the image with only this set of 'scale' parameters from the image server. This is a feature which is necessary for Volto.

This gave me a good slap in the face on my own 'autoscale' idea. Supporting "@@images/image_field/autoscale_600x400" can only be trusted in server side templates, for separate frontend you need the thumbor like solution.

What I have found so far by talking on sprints about image handling is that we have a huge pool of possible improvements and new features and that it is also very easy to cherry pick implementions for you personal or organisational image handling itches that are insufficient or maybe even blocking other functional improvements.

I asked around at the end of the Open Space if end users/editors had more features besides the decoupled image server setup

  • one of them was better copyright/licensing support in Plone for image material.

  • Markus main issue we talked about in Innsbruck was the selection of image scales to editors in tinymce. A lot of scales are meant to be used 'hardcoded' in templates or lower level (the social media scales for the og:tags metadata for eample) and you'd like to offer the editor in tinymce only a subset of scales for the wysiwyg text area. Maybe you even want to customise availalbe scales for different rich text fields. (This is traditional Plone, iirc Volto's approach is to have separe Image blocks and not insert images into richtext).

  • Limit image upload dimensions and/or put validators on the image uploads. Image dimensions or image file was was possible in Arcehtypes but no longer in Dexterity image fields. For this is also some work done already or re-implemented (edit: it is, see list below)

  • My 'personal' customer projects wish list inside Zest: I want to automatically run a focal point detection after an image is uploaded and I'd like to rip out color profiles, run optipng and/or scale down the 'source' image to max dimensions (a variation on the dimension validator). This all boils down to making the upload phase 'pluggable' with image modifiers.

  • Focal points: you could do this automatically but a long standing request where we look at CastleCMS all the time is a focal point editor where editors can manually set and optimise their focal point. But why not both :wink:

  • The whole mess of image formats. png, gif, jpg, webp. If you know the frontend you render to support webp, serve the webp version of an image.

  • svg handling. (this already has improved quite a bit in Plone 5.2)

The past 4-5 years a lot of add'ons have been released that all adress some of these itches. They all tend to get in each others way because the current image handling pipeline isn't extendable enough so that you can safely experiment in your own Plone projects without monkey patching or overriding at specific point in code.

  • plone.app.imagecropping (focal point editor where editors can create several crops on the same image)
  • colllective.autoscaling (loop over all image fields in your Plone site and scale them down)
  • collective.externalimageditor, Products.imageEditor (basic image edit functions for editors)
  • medialog.tinymceplugins.imagestyles (add more selectable styles to images)

There also has been work on making image handling more 'pluggable' in the last 12 months:

(plone.namedfile is where the image 'ingress' magic happens nowadays).

We also had the support for HiDPI images added to Plone 5.1, which our former coleague Diederik championned together with @mauritsvanrees (https://github.com/plone/Products.CMFPlone/issues/1483 for the PLIP). A nice side effect/requirement of this PLIP was that all places where images in Plone were inserted had to be changed to using the scale.tag() function so that we can generate html5 source sets and/or make any other kind of html structure around the image pluggable.

But again: this only works in our Server Side Rendered (SSR) traditional Plone setup, and is a dead end in if you want to properly support support 'decoupled' frontends like Volto, then you need a well defined API, handle the security/ddos issues, etc.

Supporting Volto means extending plone.restapi and at least solve some critical issues there are now that a request to plone.restapi triggers the generation of many image scales when only one is needed. @tisto just explained this in the previous post.

It can be tempting to say "noo, this or that feature is not useful or 'wrong', because I/my projects don't need this". And then somebody else describes their setup and in their use case it's a vital part or at least very useful. Reality is that we all have a different subset of requirements for image handling depending on the projects, end users and customers we a dealing with.

Maybe you don't need to downscale images when they are uploaded. You have 'power editors', but my end users are downloading the 6000x5000 images from a company asset management system and insert them 1:1 in a carousel item :open_mouth:

Soooo, a lot of information and a large problem space. Current situation:

  • Pluggability on image upload/validation has been implemented partially but maybe could be generalised to also support image transformation on upload.
  • pluggability on image output is a big theme. We could jump straight to the Ferrari 'external image server' use case/setup as our ideal solution, but that can take quite some time to implement.
  • The feature set in 'core' Plone has stagnated for a long time (mainy because of the complexity)
  • There is low hanging fruit here in community add'ons and idea's
  • A subset of requested features are orthogonal to the upload or delivery phase and are more metadata or editing on the content management level (licensing fields, tinyme integration, focal point editor & storing, exif metadata)
  • Volto needs some urgent fixes and improvements in Plone/plone.restapi.

My hope is that if some 'smart' people can look at the current code base and improve
upon the pluggabilty which is already there on the developer level. Then the current image handling in Plone will be the default 'plugins'. Such a re-organisation will make it easier to start experimenting not only with Thumbor but also allows developers to experiment with smaller features at the different stages without directly 'overwriting' or monkey patching.

I still have my 'history of image handling' in Plone write up from February. I can edit it a bit and post here on community.plone.org. I have the impression there are other features and wishes in the community that are not yet visible and I'd like to get more 'world wide' feedback. My intention was to ask around again at PLOG and collect more feedback/organise discussions in Sorrento but we have to switch to online now, which we had to do anyway at some point to get this 'global' community feedback.

/brain-dump-end

4 Likes
7

Thanks @tisto and @fredvd for the comprehensive explanation. This really gives a lot of background and insight into what kind of use cases should be taken account when touching these packages.

I'm tempted to scratch my own itch next week and see if I can fix any issues with Volto use case, in a few days, by "trying to improve Plone itself".

Why we could still scale images withing Plone:

  • Python can do it. Ecosystem is good and libraries are fast enough.

  • Scales are blobs. Plone is good enough in serving blobs.

  • Cache invalidation is never fun. With more deduplication services even less so.

And I believe that there are low hanging fruits to pick to make scaling more performant:

  • Disable scaling on demand (write on read). Scales should be populated on image field value change. When scale is missing, it should do temporary redirect on the nearest available scale.

  • When all scales are generated at once, they could be generated in parallel with multiprocessing.

  • Remove the current auto-expiration of image scales.

  • When scales don't auto-expire and are only auto-updated after change, it is safe for custom code to listen to change events, asynchronously update the initial scale with custom version, and purge cache.

Also I want to make scale UUIDs deterministic. It was not fun to discover that, thanks to the current auto-expiration with random UUIDs, images just disappeared, because the images referenced from your cached HTML no longer exists on the server...

Finally, I question the need for exact pixel scales on demand: src-sets made from "named scales" with modern object-positioning and object-sizing CSS provide pretty good responsive experience. We do also focal point with object-positioning (or background-positioning).

My desire to disable the scaling on demand is probably the most controversial and breaking change proposed. But for me, scaling on demand seems to be the beginning and root of all evil here.

8

Yes, I really would prefer keeping the scale generation on the fly like it is now. I do not see any advantage in polluting the Data.fs blobstorage with tons of files that might not be used.
Remember that we also support responsive images here...

2 Likes
9

In order to be clear what's the Volto problem, just because it can do that in an async way :slight_smile:

Volto site with a homepage. This homepage has 40 teaser elements, each one, having to show a title, description and image (lead image). So you access it, (cold database) then at the same time, the 40 objects are waked, then requesting all the scales (because p.restapi returns all of them as content information for every image field). Then the scales are done on the fly (provoking the dreaded write on read) for each one of the scales for each one of the 40 elements. At the same time.

Imagine a content that has more than an image...

So it would be more than enough for me, if:

  • The images are not done on the fly, they are pre-baked on save, and cached somewhere (I don't care if they are or are not blobs, whatever). So waking the object does not create all the scales of all the image fields of the content type every time if they are not in memory.
  • The shorthands @@images/image_field/name_of_the_scale works bypassing the object (lets say you come from a catalog search, and you want to access the image without waking the object), and then serve the pre-baked scaled image directly.
  • If someone creates a new scale via programmatically or via control panel, then when the shorthand is used or when accessing the object scale, the system should be smart enough to generate the scale on the fly, save it, then serve it.
1 Like
10

For those reading along and want to know what is happening in current Plone without plone.restapi: I have just uploaded one image in a fresh local coredev-5.2 checkout and added it at the bottom of the homepage with scale large.

  1. you upload the image in an Image content type. 2 blobs get's created, the original image and one scale because after uploading the 'view' template requests the large scale on the image.

find . -type f
./.layout
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x67/0x03d74473c9f90966.blob
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x78/0x03d74473ceb53a55.blob

  1. I go to the homepage, edit it and insert my image while selecting scale large. Still in the tinymce editor 2 more blobs are created on the filesystem.

find . -type f
./.layout
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x67/0x03d74473c9f90966.blob
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x7e/0x03d74474b76bfcaa.blob
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x80/0x03d74474cfb98422.blob
./0x00/0x00/0x00/0x00/0x00/0x00/0x1b/0x78/0x03d74473ceb53a55.blob

This is a bit weird and scary. when I copy all 4 blobs to temp and rename them to an extension with .png there is the original image and 3 'large' scales' in separate blobs. :open_mouth:

ls -alh /tmp/blob*
-r--r--r-- 1 fred wheel 1.0M 17 Apr 16:20 /tmp/blob1.png
-r--r--r-- 1 fred wheel 414K 17 Apr 16:20 /tmp/blob2.png
-r--r--r-- 1 fred wheel 414K 17 Apr 16:19 /tmp/blob3.png
-r--r--r-- 1 fred wheel 414K 17 Apr 16:19 /tmp/blob4.png

Tinymce generates a link to the image with the image scale and uses ./resolveduid as well:

<img alt="" src="../resolveuid/d9ad9cbcc3254ac385c17274a600018a/@@images/image/large" class="image-richtext image-inline" data-linktype="image" data-scale="large" data-val="d9ad9cbcc3254ac385c17274a600018a" />

  1. When I save the page, I think some transform picks up the the resolveuid link and another sees @@images/image and converts the link to a semi permanent uuid-like version:

<img alt="weak vs moderate caching" class="image-richtext image-inline" src="http://localhost:8080/plone/testimage.png/@@images/c767fd18-b8c6-49bd-b0c8-62e2eb731d31.png" title="testimage" />

This html can get cached by a reverse proxy like Varnish and the first download to http://localhost:8080/plone/testimage.png/@@images/c767fd18-b8c6-49bd-b0c8-62e2eb731d31.png can also be cached.

[ edit/addendum:

I just tested hidpi support as well on Plone 5.2 master and it is completely broken. My first thought was that the extra blobs were the hidpi scales, but hidpi support is disabled by default, after enablig it, no cookie :frowning: hidpi works fine in Plone 5.1.6. HiDPI image generation is broken in Plone 5.2, works in Plone 5.1 · Issue #3085 · plone/Products.CMFPlone · GitHub ]

2 Likes
11

Sounds like a great plan!

1 Like
12

Thanks @sneridagh to pinpoint the issue

I bet it returns the URL/path to the scales in some json and not the actual scale content.

So I would say it is enough to just separate the logic that generates the scale URLs from the one that actually creates the scales.

The image image will be actually scaled (with a really sane write on read) when the scale content is needed...

In this way we make everybody happy, don't we?

Do not forget that scales might change at any time by fiddling with the site settings.
What should we do then? Iterate over all content to regenerate all the scales?

13

So your content creator will came to you saying: I've just uploaded a 40 images teaser and went to view it immediately (or the browser redirected him at the view) and guess... errors or empty images because Plone is still calculating all the scales.

The correct way to handle it is like they do in youtube, stage and wait until it is done, then publish/save/whatever. 1 way form will never solve the user experience. If you're installing something, there's a nice progress bar, a clear "please wait until..." message or "pick up a coffe and return later".

Image the same process when you upload an image in wordpress. You see it loading and you can act when finish (or when you click on a uploaded and elaborated image).

2 Likes
14

It does, the download URL, but to know which one is that, it has to create the scales first.

image: {
 content-type: "image/jpeg"
 download: "https://volto.kitconcept.com/api/test/slider_pyranometer.jpg/@@images/a99874e0-68be-    4c6b-8795-2aa9904222e1.jpeg"
 filename: "slider_pyranometer.jpg"
 height: 350
 scales: {
 icon: {,…}
 large: {
  download: "https://volto.kitconcept.com/api/test/slider_pyranometer.jpg/@@images/17588e0c-df0a-4c02-9376-904282359f19.jpeg"
  height: 140
  width: 768
}
15

If someone creates a new scale via programatically or via control panel, then when the shorthand is used or when accessing the object scale, the system should be smart enough to generate the scale on the fly, save it, then serve it.

16

So I think it is superclear to me that if you could have the URL
https://volto.kitconcept.com/api/test/slider_pyranometer.jpg/@@images/17588e0c-df0a-4c02-9376-904282359f19.jpeg
without having to create the scale and postpone the scale creation to the first time a browser fetches the resource with your browser (thanks to the write on read) everybody will be happy.

Am I missing something?

17

Yep! that would be great!

1 Like
18

Although that won't solve the write on read the first time the access happens, but that might be acceptable, since it will happen only once.

19

Well, write on read in this case is a super nice feature, as saying before it copes with the fact that you can change the scales in portal settings without the need to rewrite all your scales.

20

With <picture/>-tag the grid block does not need to know what kind of scale it needs. You only need to list all the available scales, and tell browser the relative or absolute size of the grid block on different view port sizes. Then browser picks the one that fits best.

3 Likes