Sleeping without a Pillow

Sometimes the bitter PIL,
then a comfortable Pillow.
Wrote on read against our will.

Now rest in peace —
Thumbor takes it from here.

We're upgrading a Plone 5.2 site to 6.2 right now. The site has tons of images — 230 GB of blobs. I'm running it on the new-stack with zodb-pgjsonb and plone-pgcatalog.

The PITA about Pillow in Plone: every image scale loads the full original blob into memory, resizes it with Pillow, and writes the result back into ZODB. Each scale seems to touch 3 small objects in the database. Multiply that by every image, every scale size — and you understand why we have 230 GB blobs: it produces so much database churn. The data never sleeps, because Pillow keeps waking it up.

So we cut Pillow out of Plone entirely.

plone-pgthumbor replaces image URLs and metadata with direct links and @@images scaling with a 302 redirect to Thumbor. No more Pillow in the Plone process, no more write-on-read. Thumbor does the scaling on the fly, supports smart focal point detection, and caches the results itself. Plone never touches the image bytes. Security is layered: all Thumbor URLs are HMAC-signed so nobody can request arbitrary transformations, and for non-public content the thumborblobloader's auth handler calls back to Plone's @thumbor-auth REST service — a single PostgreSQL query against the allowedRolesAndUsers index, no ZODB object loading needed.

But Thumbor needs to read the original blob. That's where zodb-pgjsonb-thumborblobloader comes in — a Thumbor 7 loader that reads blobs directly from PostgreSQL's blob_state table (or S3 if you tiered them out). Async connection pool, local disk LRU cache, no ZODB, no unpickling. Just bytes from PG to Thumbor.

And with plone-pgcatalog the catalog no longer chokes on all those scale annotations either. PostgreSQL handles the queries — no BTree bloat from thousands of scale entries that nobody ever loads anymore.

:warning: This is all early code, do not use yet in production, except if you know what you're doing!

Try it out

Want to see it in action? There's a Docker Compose tryout that gets you a full stack (Plone 6.2 + Thumbor + PostgreSQL + nginx) in minutes. Upload an image, right-click it, and inspect the URL — you'll see the signed Thumbor redirect in action.

Full documentation: plone.pgthumbor 0.1.0 documentation

The before and after

Old chain New chain
request request
Plone Thumbor
ZODB load PG / S3
Pillow resize transform
ZODB store cache
serve serve

Plone just creates a correct stable URL or redirects.

The image data finally sleeps without a Pillow.

4 Likes

@jensens on Claude Steroids ... I like! :wink:

2 Likes

right, :grinning_face_with_smiling_eyes:

@jensens I’m still very impressed, this approach would be valuable even without pgjsonb.
My only concern is the redirect: that extra round-trip can significantly impact page responsiveness and performance.
We need a way to bypass it. I was thinking of a mechanism similar to X-Sendfile or X-Accel, or by leveraging the WSGI pipeline or any other brilliant idea you might have, which, at this point, we’ve almost come to expect! :wink:

1 Like

Usually URLs are created with direct links most of the time, just if you access @@images/FIELD/SCALE it redirects. This was discouraged to use long time ago.

Usually URLs are created with direct links most of the time, …

Understood. That makes perfect sense. Thx.