Relstorage: Still used?

I'm interested in possibly using RelStorage, but wonder how wise that might be given it hasn't seen any commits since December 2014. Does anyone have feelings on the viability of the project and how good/bad of idea it would be to use it for a new site?

Independent of the viability: ask yourself if you really need it. We've been running big Zope and Plone projects for almost over 15 years and never need Relstorage. Unless you have serious arguments for using Relstorage: stay mainstream...less moving parts, less problems.

-aj

RelStorage is still used and quite stable. We are using it for our Zoplo deployments in production. If you have many authenticated users working on a Plone site, relstorage can be a performance booster. I'm not totally sure, but AFAIK it leverages row-level locking, which is not possible with FileStorage. I'm not an expert on this, @jensens is...

We (Zest) use it for one site, Plone 4.2 moving to 4.3, where the hosting party wanted everything in Postgres. Working fine for us.
But yes, all our other sites use standard filestorage. Also working fine. :slight_smile:

That's a good point, and we've run production sites without RelStorage. The reason for looking at the possibility is leveraging our existing HA postgres setup.

We are using it indeed heavily. RelStorage is really mature. We're using 1.6.0b3 - don't let the beta blind you, its defacto more stable than most software.

RelStorage in history free mode is slightly faster. On the client side it utilizes a memcached shared between instances for DB buffering. So it makes sense specifically for larger installations with lots of objects and instances. It worked fine with 42mio objects for me, just packing was a problem. This was solved meanwhile (internal and with an addon). Using the poll-interval may help you in massive read only scenarios to get load off the DB.

I know some organisations with larger sites using RelStorage. You may need to tune your PostgreSQL DB setup according to your needs, so having a zope and pgsql expert working together is a good idea. This needs more planning and time, but is from my experience worth the effort if you have a larger site.

If you have specific questions, please ask I will try to answer.

1 Like

We swtiched from relstorage to ZRS. Relstorage was working ok ish but we probably still didn't have it optimised. ZRS pretty much just works and less moving parts. But I'd have no problems using relstorage again if I had to for external reasons.

I should add that the zodbpack and zodbconvert scripts bundled with RelStorage are really useful for various things. For example, we use zodbconvert to make FileStorage backups of our RelStorage (which take longer than pg_dump/pg_restore to use, but are smaller) to complement our pg_dump backups.

After initial time investment in RelStorage buildout, the only incremental cost over ZEO we have had has been small time investment in regular software updates, which is due to building own Postgres, not RelStorage itself.

RelStorage pros:

  • Many more avenues for HA possibility (not that choice is in itself better).
  • Multiple backup avenues.
  • Marginally faster than ZEO.
  • You don't need to worry about ZEO having access to your SOFTWARE_HOME, RelStorage server-side is just (dumb) RDBMS.
  • DBA-friendly for larger organizations.
  • If you have SQL chops, you can trace back-references to objects by oid of target (generated during pack, so can be stale, but still useful).
  • TLS support: you need a client on another box to connect an encrypted channel over TLS, using certificates for authentication. This is very nice, and cannot be done in ZEO without wrapping in stunnel process.

RelStorage cons:

  • More work (up-front).
  • More complexity (dependencies, buildout work).
  • More learning curve for database administration for average developer (backup, tuning, HA, etc).
  • Storage: multiple what FileStorage takes up, excluding BLOBs, by 1.5 to 1.6. PostgreSQL indexes (even when using auto vacuum) just take up more space). pg_dump is not as tight a binary backup as FileStorage (though you can back up RelStorage to a FileStorage copy).
  • Commit-lock (deadlock) issues; I am a bit of a deadbeat and did not submit a fix to use PostgreSQL 9.3+ commit lock timeout support outside of a github fork I am temporarily using.

Toss-up:

  • Tuning: complexity vs flexibility? Depends on your angle or experience.

Sean

4 Likes

Somebody buy this guy a beer :slight_smile: ...once he puts this in documentation someplace :smiley:

1 Like

Please note that RelStorage is being actively developed again and now supports ZODB 4 and 5. See https://github.com/zodb/relstorage/blob/master/CHANGES.rst for a list of the many recent changes.

also https://pypi.python.org/pypi/zc.zrs/2.5.3 will be updated to use 5.0? That would be a great thing to have zrs in Plone.

How many people are running Plone on ZODB 4/5?

@yurj Has anyone tried? What is the problem?

1 Like

Nobody so far I guess since we need to finish https://github.com/plone/Products.CMFPlone/issues/1351 first for people to be able to use it.

Thanks to everyone for their replies -- very much appreciate it!