Database Conflict Errors: ZPublisher vs. Waitress

Zope
1

Hi everyone,

we have recently updated our system from Plone 4.3.20 -> Plone 5.2.3 (still under Python 2.7.x) and used ZPublisher until now.

However, due to an increased number of database conflict errors that occurred on the UI, we decided to change to waitress and it seems that it works a lot better with the conflict resolution and is much faster!

But how can this be that it is such a difference between these two servers with regard to conflict resolution?

This is an error that occured very often to us under ZPublisher that also popped up to the UI and that completely vanished now since we changed to waitress:

Traceback (innermost last):
  Module ZServer.ZPublisher.Publish, line 151, in publish
  Module ZServer.ZPublisher.Publish, line 393, in commit
  Module transaction._manager, line 257, in commit
  Module transaction._manager, line 135, in commit
  Module transaction._transaction, line 282, in commit
  Module transaction._transaction, line 273, in commit
  Module transaction._transaction, line 465, in _commitResources
  Module transaction._transaction, line 442, in _commitResources
  Module ZODB.Connection, line 692, in tpc_vote
  Module ZEO.ClientStorage, line 752, in tpc_vote
  Module ZEO.asyncio.client, line 764, in call
  Module ZEO.asyncio.client, line 743, in call
  Module ZEO.asyncio.client, line 756, in wait_for_result
  Module concurrent.futures._base, line 462, in result
  Module concurrent.futures._base, line 414, in __get_result
ConflictError: database conflict error (oid 0x0861df8c, class BTrees.IOBTree.IOBucket, serial this txn started with 0x03dd91b7edf040bb 2021-01-25 09:27:55.766833, serial currently committed 0x03dd91bf28989b22 2021-01-25 09:35:09.514715)

I always thought that conflict resolution is part of the transaction manager which tries 3 times until it raises an error...

Maybe someone can bring some light for me with this behavior.

Thanks a lot
Ramon

2

I do not see any relationship between both.

3

When you use Zope (or Plone), you always use ZPublisher (it is responsible for url traversal and the publishing of Zope/Plone objects). You mean ZServer (instead of ZPublisher), it is an alternative to waitress.

Conflicts do not depend on the IO component (ZServer or waitress) you are using. You should not see significant differences.

4

Thanks for your answers.

I have seen the functions ZServer.ZPublisher.Publish.publish and Zope.ZPublisher.WSGIPublisher.publish.

Therefore, I meant the first function that is used when wsgi=off and the second that is used when wsgi=on.

It seems to me that the second one uses a different approach (transaction manager?) that is more robust regarding to ConflictErrors than the first one, and was curious what it might be.

Thanks again,
Ramon

5

I have just got bitten by it :smiley:

It seems that the first item reported on the issue Zope#413 still affects ZServer, i.e when WSGI is disabled.

Long story short: HTTPRequest.retry_max_count was made configurable, but it was buggy and defaults to zero, meaning do not retry on conflict errors.

This problem was solved, but only for the WSGIPublisher. The buggy line is here: ZServer/Zope2/Startup/handlers.py . It is setting retry_max_count in the module, not in the class.

This still affects Plone 5.2.12 when using ZServer (i.e no WSGI). However, ZServer is not supported anymore and its Git repository is read-only.

1 Like
6

@rafaelbco You can always request a repo to be un-archived via opening an issue in Issues · zopefoundation/meta · GitHub