Sessions configuration: Request for comments, corrections etc

Plone Support Customizing Plone
1

I'd like to improve my Zope/Plone setup by using Beaker and/or Memcached, and I couldn't find a coherent documentation for this topic.

So here is my writeup; I hope for comments and corrections, especially for the nonsense parts.

Sessions configuration

In the default configuration, session data is stored in the ZODB; this means, whenever someone logs in to your site, there will be data written to the end of your Data.fs file (we ignore RelStorage here, for simplicity). This doesn't perform well, and it doesn't scale well.

Another drawback is: Whenever your object database is mounted read-only (perhaps because some long-running data conversion you'll like to complete before people apply their own changes), no logins are possible.

ZODB mountpoint

In your various parts/instance/etc/zope.conf files, you'll likely find some section

<zodb_db temporary>
    # Temporary storage database (for sessions)
   <temporarystorage>
     name temporary storage for sessioning
   </temporarystorage>
   mount-point /temp_folder
   container-class Products.TemporaryFolder.TemporaryContainer
</zodb_db>

Obviously, “name temporary storage for sessioning” is not a working configuration; furthermore, this would imply the /temp_folder to be written to a zodb_db as well (another one, but still); this is not what we want to achieve.

Let's try something else.

Beaker

The simplest possibility to separate the session data is to use Beaker. In the most basic case, Beaker can write session data to a directory in the file system; like the commonly used ZODB (via ZEO or RelStorage), this can be used by all clients of our Zope instance.

In the buildout script, add the Products.BeakerSessionDataManager module to your eggs value.

This requires collective.beaker and contains the “zcml slug” which is mentioned in the collective.beaker documentation; it is needed as well to make Zope/Plone use beaker-based sessions itself.

  • Question: Products.BeakerSessionDataManager v1.1 requires Zope2, so it apparently won't fit Plone 5.2, which uses Zope 4+.
  • Is it possible to use beaker in the same or another way with Plone 5.2+?

To have beaker how to store the session data, add some configuration to our buildout script, like described here; e.g.:

zope-conf-additional =
    <product-config beaker>
        session.type            file
        session.data_dir        ${buildout:directory}/var/sessions/data
        session.lock_dir        ${buildout:directory}/var/sessions/lock
        session.key             beaker.session
        session.secret          secret
    </product-config>

(„secret“ of course being a secret string unique to your Zope instance)

In a bash session, you might create the new directories by calling

mkdir -p var/sessions/{data,lock}

Take care them to be readable/writeable to the system user you use to run Zope.

After rebuilding and restarting your Zope instance,

  • open the ZMI,
  • delete the existing session_data_manager object in the Zope root,
  • and add a Beaker Session Data Manager instead (which will have that same id session_data_manager as well).

When users login, there should now data be written to the …/var/sessions/{data,lock} directories.

You can try as well to run your instance with the ZODB mounted read-only and see whether it is still possible to login.

memcached

One step further, you could run a Memcached server to manage the session data.

It is possible and common to configure beaker to use memcached as a backend; alternatively, you can omit beaker and use the lovely.session product instead. This will in turn use lovely.memcached to talk to your memcached server.

If you don't have a memcached server already, install it; you might use your system's package manager, e.g.:

yum install memcached

If no such package is available for your system, or if it is too old, have a look at https://memcached.org/downloads, and get the source package.

  • (Question: On my system, this installs memcached-1.4.15-10.el7_3.1.x86_64, which is obviously maintained, but based on version 1.4.15, released on 2012-09-03. Should I compile memcached myself to get a more current version 1.5.12? Obviously I lack some needed Perl package to make „make test“ succeed.)

(…)

2

but Plone itself and most plone plugins don't actually store anything in sessions to my knowledge. So unless your code specifically uses sessions then you might be optimising this for nothing?

Plone itself does store the last login date in the ZODB but not in the session so there will be a write on login regardless. (it would be nice if there was a way to prevent this or store it somewhere else. It can be a problem on a large intranet with lots of logins).

Also you neglect to mention that you don't have to use memcached and beaker to remove sessions from your main zodb. You can simply use a temp storage on you zeo server and share that with all your zope instances. Less moving parts.

3

Thank you for your answer. It's quite clear you know much more than me about this stuff; this is my effort to tickle some coherent information out of the knowing.

So, the last login date is written to the ZODB; meaning, to the user object, right?
This would mean, to allow logins to a read-only Zope session, it is necessary to use some PAM which stores user information elsewhere, e.g. pas.plugins.sqlalchemy.

Perhaps I was mislead by that Memcached as a session storage section on the "Performance tips" page. Perhaps there should be added a sentence like the following (RFC, again)?

However, if all you want is to avoid ConflictErrors and/or allow logins to a read-only site (which would require you to store your user objects elsewhere as well), and you don't use an extension product which stores interesting data in the session, you can simply use Products.BeakerSessionDataManager instead.

4

Well, first thing I mentioned was that <temporarystorage> thing in zope.conf. I couldn't find any documentation about this, and from looking at the sources, it appeared to me this to be an outdated option; thus I concluded, "Let's try something else."

Nothing is simple unless it is documented.

So, how would this be done?

5

correct. It's due to this function.

To be honest it would be nice if there was a setting to turn this off. I don't think knowing login_time is that useful and is perhaps best handled by external plugins. It shouldn't require switching to an alternative user storage.

yes its badly written but is telling you that read-only site requires an alternative user storage.

yes maybe its badly documented. The config you mention is not for mounting a ZEO based storage but a local storage per instance which you can't really use in a ZEO setup unless you have a load balancer with sticky browser sessions.

This touches on using ZEO for session storage but doesn't give an example https://docs.zope.org/zope2/zope2book/Sessions.html#instantiating-a-transient-object-container
but in any case, sessions are not your problem it seems.

6

True. My idea was to

  1. get the session data out of the ZODB (for ConflictError / performance / growth / read-only login reasons)
  2. proceed to Memcached-based caching (the "(…)" part), and
  3. configure collective.beaker to use the memcached backend as well.
7

have you actually worked out if you are storing any data in sessions?

8

I don't think so. I think you're right, and session performance is not my problem.
It would be much more interesting to use Memcached for other purposes.

9

+1000 - we even do not need a PLIP for this, it is a simple enhancement. In fact a new registry entry in the user-groups settings (Products.CMFPlone), an upgrade step (plone.app.upgrade) to load the new setting in old sites and a condition in the mentioned code would be enough. Not very complex to do. Who takes care of it?

10

We have some old, evolutional grown sites making heavy use of sessions and collective.beaker just does the job. On multi-server/docker environments memcached or redis as beaker backend are just working.

11

I suspect it might require a bit more as there could be UI that replies on sorting by last login. It would be nicer if that was disabled also to give less confusion. But you are right. simple PR would be ok.
@tobiasherp willing to mentor you on your first Plone contribution that would solve your immediate problem and anyone else who comes across this problem in the future...
even added a ticket - recording last_login on users should be optional · Issue #45 · plone/Products.PlonePAS · GitHub

12

@djay I do not know any place in Plone where this happens. But I may have overseen something: Which template do you mean actually?

13

Thank you, this is very welcome!
In fact, it won't be my very first; I have one in plone.recipe.zope2instance already ... But no "real one" yet.

One possible use case is to detect whether a freshly (self-) registered user has ever logged in at all. We have such a check in some old and ugly code ...