Image scales, conflicts and PIL errors

Plone Support Administering Plone sites
1

Hi all! :wave:t4:

we did a relaunch/face lift of our website https://www.freitag.de but since then we are seeing a few interesting tracebacks (see them below).

One thing that we did change on the new website is the scales, so we use almost all the upstream Plone 6 scales plus a few custom ones:

<element>giant 3000:65536</element>
<element>huge 2000:65536</element>

<element>portrait_lead 65536:1000</element>
<element>portrait_teaser 65536:600</element>
<element>portrait_mini 65536:200</element>

So, a few ones for really big images and a few ones that are portrait oriented (the width is set to its max so the height is the scaling factor).

Could it be that because image scales are stored in the annotation storage as fieldname-width-hash we see the raise of image scale storage conflicts? :thinking:

If the same image is scaled for the 3 portrait oriented scales, the key for the storage uses the same fieldname-width...

And that could also be the PIL related error? If the image stored is meant to be big, but is actually small, PIL runs out of bytes to read and complains? :thinking:

I slightly changed the width's of the portrait oriented scales to see if the problem goes away :four_leaf_clover:

Anyone noticed that?

Tracebacks

Our dear conflict error:

ConflictError: database conflict error (oid 0x0143adf2, class plone.scale.storage.ScalesDict, serial this txn started with 0x03fba8f7db3e5e11 2024-09-30 20:39:51.385224, serial currently committed 0x03fba8f7dd2ce922 2024-09-30 20:39:51.837992)

PIL traceback:

ValueError: I/O operation on closed file
  File "/plone.namedfile-6.2.3.post0-py3.11.egg/plone/namedfile/scaling.py", line 352, in handle_image
    result = self.create_scale(
  File "/plone.namedfile-6.2.3.post0-py3.11.egg/plone/namedfile/scaling.py", line 325, in create_scale
    return scaleImage(data, mode=mode, height=height, width=width, **parameters)
  File "/plone.scale-4.1.2-py3.11.egg/plone/scale/scale.py", line 78, in scaleImage
    with PIL.Image.open(image) as img:
  File "PIL/Image.py", line 3240, in open
    fp.seek(0)
2

The hash uses all the parameters.

If it happens only when referencing new images on a page and happen only in this case, it is quite normal.
I think that conflicts can happen with new images if the image is quite big and the page get multiple requests that generates the scales.

The error here is that a file is open and then closed, and another thread is trying to work on it and find it closed. This is a race condition, the transaction abort and in the meantime the other thread has already completed the task.

3

The conflict error is somewhat surprising because ScalesDict includes custom conflict resolution code which is supposed to help resolve them: plone.scale/plone/scale/storage.py at master · plone/plone.scale · GitHub

Yuri's explanation for why the file would be closed sounds plausible to me.

4

You need to make sure this version of plone.scale is also being used in the Zeo server. Otherwise the conflict resolution will not be called.

5

@rafaelbco I think they are using Relstorage, not ZEO.

1 Like
6

Thanks for all the answers and tips!

Yes, we use relstorage.

I think that I finally understood the problem... we changed the image ratio of our images from 3:2 to 16:9, so we ran a script to cut them.

Unfortunately, when saving them back, there was a silly conversion error between PIL and NamedBlobImage...

At least that's what I suspect, looking at the +600 broken images we detected via SEO tools.

Fortunately I had a backup at hand from before anthem cut, and I could recover those images, I still need to re-upload them and double check they indeed fix the problem :smiley: