Image scales broken on edit

Plone Support
1

It should first be noted that Plone out of the box does not assign a workflow chain for Images, so assume we sign simple_publication_workflow. I add an image and use it in a page, where the link with default Large scale is http://localhost:8080/jellyfish.jpg/@@images/98730230-6876-4780-820a-b2ea8d6a4138.jpeg. I believe the 987~ id is a name assigned by PIL for the scale. I then change the workflow state to private. The above now returns a 404. http://localhost:8080/jellyfish.jpg itself is fine and if I view it in the page editor it will happily get a new scaled image for me.

I've tried this with local builds of 5.0.5 and 5.1.4, with my custom eggs removed, and had the same error.

Under the assumption that this had something to do with how scaled images are stored, I looked into plone.namedfile.scaling. I see that it is passing a parameter to AnnotationStorage with the modified time (context._p_mtime) for the sake of invalidating caches. This looks promising - it would make sense that changing workflow might update modified time. I commented out passing in self.modified to test, and that seems to work. I've got http://localhost:8080/ootb/jellyfish.jpg/@@images/27048724-76f9-4476-b3a3-90c7988f5a7e.jpeg and going through several wf state changes that image is still valid.

Frustratingly, I cannot get this to repeat on http://demo.plone.org. The scale id remains sticky and valid through wf changes there. I don't see anything that should be obviously different about that environment. It also doesn't make sense because I can see the modification date has been updated when I change workflow state there.

2

Ok I see that's on 5.1.5 not 5.1.4 so I upgraded to that. Doesn't seem to make a difference for me. I'm at a loss, can anyone provide some insight?

3

Looking at this more, the behavior I'm getting seems logical although not ideal.

We're hitting the publishTraverse method with @@images and it detects that "98730230-6876-4780-820a-b2ea8d6a4138.jpeg" is a uid https://github.com/plone/plone.namedfile/blob/4.2.6/plone/namedfile/scaling.py#L297. But this uid no longer exists in storage. If I step back and look at what is deleting the scale from storage, it's this line in plone.scale https://github.com/plone/plone.scale/blob/master/plone/scale/storage.py#L183. So because self._modified_since(info['modified']) is true it deletes it. I still think it would be better to only see if the image has changed and not the metadata but I don't know if that's possible.

My other question is why does html_to_plone_outputfilters_html change the image address from @@images/image/large to @@images/{uid}? The orig source from the wysiwyg is using that preset, it's actually the output filter that is converting it to a uid (rescaling it?). Why?

4

I think I can simplify the problem greatly. I did the following steps on a new out-of-the-box Plone site

  1. Put the Image type in a workflow
  2. Add "/view" to the View action of Image in portal_types. This page displays a scaled version of the image
  3. Change state of the image

On the view page you can see that every time the wf state is changed, the image source URL changes because it has a new UID. On demo.plone.org, the old scale caches remain and this lets pages continue to reference them. On a 5.1.5 environment I've made the cache is deleted. To be honest I don't understand why they are NOT deleted on demo.plone.org because of this https://github.com/plone/plone.scale/blob/master/plone/scale/storage.py#L183 but that is the behavior I want.

5

demo.plone.org is being served from CloudFlare. It could very well be that the image is removed from Plone, but the caching parameters for these scales is set to 24h or longer and Cloudflare happily still serves them for this period. The same would go for a Varnish reverse proxy if you do the caching in your own stack.

6

Thanks Fred, that makes sense. If that's true it sounds like I don't have many options here. I could restrict metadata changes (workflow). Or monkey patch out where it deletes that scale from Plone - it might be that workflow changes are rare enough that having excess storage isn't really a big deal?

7

That is sort of a deployment issue. They do support sending a PURGE request. This is a common HTTP verb for caches and Varnish supports this. There used to be support for Varnish via some Plone package, so that should 'just work' for CloudFlare as well, unless the image scales broke something on a conceptual level in their design.

8

Maybe Varnish should be purging it but ultimately that just means it's currently masking another problem. I am leaning more towards wanting to find a solution in the transform actually, rather than how Plone stores image scales. As I mentioned before, if you put an image in a page it's going to have a URL like ${context/absolute_url}/@@images/image/large. That URL is going to work if you change the title, or the workflow state, or even the image itself. From a UX perspective I think that's the behavior I want - I shouldn't need to go back and edit the page because I updated the image, it should just work. It is the plone output filter transform that is converting it ${context/absolute_url}/@@images/scale-uid. That output filter is trying to accomplish several meaningful things that I do not fully understand; I don't know if there was a good reason to do this or if it can be fixed without breaking something else.

9

It should not. Plone should tell Varnish or CloudFlare upon the modification event to invalidate the relevant cache, if it gets invalidated Plone-side before the expiration time is reached.

10

So the plone outputfilter transform is keeping its ~1 hour cache when an image is edited. This wouldn't be a problem if it kept the size preset as saved in the page source code, but it is converting that into a link directly to the scale id. Here's the monkey patch I'm looking at applying to plone.outputfilters.filters.resolveuid_and_caption.ResolveUIDAndCaptionFilter

def resolve_image(self, src):
    obj, subpath, appendix = self.resolve_link(src)
    if obj:  # don't change anything about images that aren't plone content
        image, fullimage, src, description = self._old_resolve_image(src)
        if fullimage:
            src = fullimage.absolute_url() + '/' + subpath
        return image, fullimage, src, description
    else:
        return self._old_resolve_image(src)

What that tries to do is see if the link resolves to an image object on the Plone site. If so, put the src back to e.g. ${image_url}/@@images/image/large instead of ${image_url}/@@images/${scale_uid}. I don't know if this will cause it to always generate a new scale, or only as needed. An alternative would be to add an event listener to image edits, find back references, and force transforms to be run on them.

11

I have this problem on a Plone 5.1.5 site. I added this monkeypatch but it did not...scratch that... it does seem to fix the problem. Thank you.

It doesn't look like it has been addressed in upcoming 5.1.6. I have not checked 5.2.