Does anyone use ZRS?

Plone Support
1

We use ZRS (GitHub - zopefoundation/zc.zrs: Replication framework for ZODB) to replicate ZODB databases with great success for some clients and and are wondering what others are using to replicate their database unless they use RelStorage or use ZRS with versions older than Plone 5.2.2.

ZRS is not yet compatible with ZODB 5.6, which is the ZODB-Version used since Plone 5.2.2. There is a open pull-request with a failing test and there may be issues with replication blobs (Reach ZODB 5.6 compatibility by dataflake · Pull Request #7 · zopefoundation/zc.zrs · GitHub).

Is there something better/newer/simpler around that we missed?

If not, @tschorr is planning to use the upcoming Zope Sprint this Friday ([Online] Zope July Sprint 2021 | Meetup) to work on getting ZRS compatible with the newest Plone versions and test replication with a large DB with a ton of blobs.

2

I used it for a real estate application based on Zope 3/ZTK some time ago. We had some secondary servers distributed over multiple data centers and providers. One reason was to limit downtime in case of failures with the primary database/data center, the other was to offload the backup process to a secondary server to free resources on the main database. We also planned to use API access on read-only seondaries, but that was never implemented.

I once had to switch hosting providers and with ZRS we had a minimal, not noticable downtime (only the time to restart the zeo server in primary mode and nginx reload to point to the new zeo clients — all done with ansible).

There is also the option to use this for Plone upgrades with large databases where the backup, transfer, restore, upgrade process would take several minutes. With ZRS you can prepare everything and at any given point in time disconnect the secondary, make it primary, run your migration, done :wink:

1 Like
3

I am using it but I also sync the blobs with rsync for good measure (see Reach ZODB 5.6 compatibility by dataflake · Pull Request #7 · zopefoundation/zc.zrs · GitHub).
That is good enough for me now even if it is far from optimal.

4

I do not see any reason to use ZRS, now that we have RelStorage.

Relational Databases like PostgreSQL have plenty of replication and scaling mechanisms, way much better maintained than ZRS is.
Additional ZRS docs are a nightmare (there are no real docs, I often looked into the code and even this is confusing work. And if you have a problem, solving it is almost only possible with lengthy trial and error session.
Since PostgreSQL is in rapid development by a thriving community its even improving from release to release with advantages for RelStorage. Jason, the maintainer of RelStorage is keen to get those advantages used by RelStorage.
Another advantage of RelStorage 3.x: a separate blob storage is obsolete and transactions with blobs involved are completed in shorter time, since there is only one place to lock. This increases the write speed.

Last but not least: RelStorage is from my experience much faster than ZEO or probably ZRS under high load and with many objects. That said I do not have recent measurements.

And in the end sysadmins are much happier with a well known PostgreSQL cluster than with this unknown thing ZRS.

2 Likes
5

I also using ZRS for a few projects, especially where a PostgreSQL DB would be overkill.

Sprinting on that topic would be. nice idea.

6

Sorry, but IMO ZRS is overkill. Postgres is just there, any modern Linux has it, or as Docker/K8s, anywhere.

1 Like
7

Additional ZRS docs are a nightmare (there are no real docs, I often looked into the code and even this is confusing work. And if you have a problem, solving it is almost only possible with lengthy trial and error session.

I always found the package README + plone.recipe.zeoserver sufficient given zc.zrs provides only 3 options (replicate-from, replicate-to, keep-alive-timeout). I find the test code very confusing, but I don't actually think the production code (< 2000 lines) is.

Last but not least: RelStorage is from my experience much faster than ZEO or probably ZRS under high load and with many objects. That said I do not have recent measurements.

So you want to discontinue ZEO in favor of RelStorage? I'd really be interested in benchmarks comparing recent ZEO/RelStorage versions in different scenarios (e.g. high write, many blobs, ...).

8

@tschorr Do you know zodbshootout ?

2 Likes
9

Yes I do know the package. If somebody uses it to prove that RelStorage should be the first + recommended storage choice for Plone, fine for me :slight_smile: But I'd prefer to have a proper roadmap for the transition from ZEO to RelStorage over being forced to switch because ZEO replication accidentally broke between Plone 5.2.1 and 5.2.2.

10

It sounds more like a bug/regression, so "someone needs to fix it" (like just do it, insert coin, hope it happens, ...).

11

No, I do not want to discontinue anything. Based on my experiences I just wont recommend to use it with larger object numbers and user bases. For smaller projects ZEO is fine. I have my doubts if it comes to ZRS.
It's FOSS: As long as there are people using and maintaining ZEO and ZRS it will live on. This is the choice of the authors and users, not of somebody on the internet who might be wrong :slight_smile:

12

There has indeed been some progress on the issue during and after the Zope July sprint, resulting in a merged PR.

So love it or hate it :wink: but ZRS is back.

8 Likes