ServerException: ('exceptions.OSError', (13, 'Permission denied')) on first load and images with /index_html view

I am upgrading a 5.0.3.1 site to 5.2.1, via 5.1.6. Python 2.7 for now until I fix all current issues and upgrade my add-ons to Python 3. This site started out on Plone 5.0 with Dexterity CTs, except for some AT add-ons (PloneFormGen, PloneTrueGallery).

Once I removed all incompatible add-ons and ran the upgrade on 5.2 and cleaned up traces of AT, I still see somewhat random tracebacks like the ones below. I was able to identify two different scenarios where this happens:

  • non-image content items, only on first load
  • image items, only some of them, when index_html is appended as the view

Here is the first scenario:

2020-05-25 14:18:29,653 ERROR   [Zope.SiteErrorLog:251][waitress] 1590441509.640.373734628229 http://localhost:9071/Plone/get-involved/default-page/document_view
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 162, in transaction_pubevents
  Module transaction._manager, line 252, in commit
  Module transaction._manager, line 131, in commit
  Module transaction._transaction, line 311, in commit
  Module transaction._transaction, line 302, in commit
  Module transaction._transaction, line 447, in _commitResources
  Module transaction._transaction, line 424, in _commitResources
  Module ZODB.Connection, line 692, in tpc_vote
  Module ZEO.ClientStorage, line 752, in tpc_vote
  Module ZEO.asyncio.client, line 764, in call
  Module ZEO.asyncio.client, line 743, in call
  Module ZEO.asyncio.client, line 756, in wait_for_result
  Module concurrent.futures._base, line 462, in result
  Module concurrent.futures._base, line 414, in __get_result
ServerException: ('exceptions.OSError', (13, 'Permission denied'))

NOTE: this has nothing to do with file system permissions of blobstorage and such. I checked ownership and permissions of everything under filestorage and blobstorage and it's correct.

It seems to happen when a given content item gets loaded for the first time, and never recurs again for the lifetime of the Zope instance. When I restart the instance, I get the same error again on first load.

The second scenario is for images. The Traceback is identical, but notice the index_html at the end of the URL:

2020-05-25 14:54:04,918 ERROR   [Zope.SiteErrorLog:251][waitress] 1590443644.920.874588936975 http://localhost:9071/Plone/hikes-events/hikes/thomas-preserve-paddle/@@images/image/index_html
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 162, in transaction_pubevents
  Module transaction._manager, line 252, in commit
  Module transaction._manager, line 131, in commit
  Module transaction._transaction, line 311, in commit
  Module transaction._transaction, line 302, in commit
  Module transaction._transaction, line 447, in _commitResources
  Module transaction._transaction, line 424, in _commitResources
  Module ZODB.Connection, line 692, in tpc_vote
  Module ZEO.ClientStorage, line 752, in tpc_vote
  Module ZEO.asyncio.client, line 764, in call
  Module ZEO.asyncio.client, line 743, in call
  Module ZEO.asyncio.client, line 756, in wait_for_result
  Module concurrent.futures._base, line 462, in result
  Module concurrent.futures._base, line 414, in __get_result
ServerException: ('exceptions.IOError', (13, 'Permission denied'))

If I remove /index_html from the URL, I can load that image without a problem in the browser. I have no idea why that /index_html is appended.

Finally, I also have a problem with scales, where some images are broken when loading with a guid scale (e.g. @@images/b02bab48-cb2c-47a3-812d-a95a223ad195.jpeg), but the image itself is fine. This might be a separate problem, but if anyone has any off-the-cuff idea, please let me know.

I also wanted to mention that Products.PDBDebugMode seems to have become useless with this wsgi stack.

ServerException almost surely indicates that the initial problem was encountered by ZEO. If you are lucky, you will find more detailed information in its log file.

I am using it (with Plone 5.2.1 and Zope 4) in version 2.0; thus, in principle, it is usable. It is possible that I have tweaked it slightly (I no longer remember the details; it was 1/2 year back).