I can confim, that the latest versions of node.ext.ldap and pas.plugins.ldap install fine on Plone 5.
But what kind of performance should we expect with pas.plugins.ldap? What kind and size of LDAP setup it should work well enough? I ask, because I've been very puzzled about pas.plugins.ldap performance with our LDAP (AD based LDAP with a lot of users and groups, but most of the groups still being smaller than 500 members).
I don't mean to whine. We are really grateful that pas.plugins.ldap finally works as a replacement for LDAPUserFolder! I'm just curious, how others have managed to adapt pas.plugins.ldap for their installations.
Some releases ago pas.plugins.ldap didn't work at all for large directories, because every now and then it loaded whole director (took a long time for 70 000 users) and also it was missing paging. These have now been fixed, and logging-in and loading groups for the current user works quite ok. Unfortunately, browsing even small groups was pretty slow.
Big part of the slowness is the design of PAS, which is unaware of lazy evaluation or iterators, and wants to load attributes and groups for each new user immediately.
Yet, while spending a this week with pas.plugins.ldap, I noticed the following and made some optimizations (most probably incompatible with the upstream):
-
At first, node.ext.ldap expects LDAP queries to be cached for it to be usable. Correct?
-
The default supported cache is memcached. After Zope profiler showed a lot of time being spent in pickling searches in and out memcached (bda.cache used both pure-python memcached-lib and pure-python pickle), I patched bda.cache to use pylibcm: libmemcached based C-optimized memcached library, which also happens to use cPickle.
-
Next I noticed multiple LDAP queries being made to retrieve the same objects from LDAP (and miss the cache). Just with different baseDN, queryFilter and attrlist. And after all the optimizations for queried attrlist, eventually the attribute behavior in node.ext.ldap fetched all attributes for the previously fetched objects. I spent a couple of days in normalizing the queries so that only mapped and mandatory attributes were queried and in the best case each object were queried only once (and then retrieved from cache afterwards).
-
The final profiling found was that (after the optimizations above) the stack spent third of it time in decoding the data retrieved from LDAP into unicode. Because the cached data was encoded, all this decoding did miss the caching. I ended up removing all decoding/encoding code from node.ext.ldap, assumed all data in node.ext.ldap to be always utf-8 encoded and only added decoding/encoding into PAS interfaces in pas.plugins.ldap.
After all this, browsing "large" groups (with members over a few hundred) still DOS the site from tens of seconds to minutes. Yet, this can be attributed to design problems in PAS, and I resolved these simply by adding limits on how many users could be "enumerated" by search, or how big groups could be browsed.
I'd be happy to hear others experiences on pas.plugins.ldap with Plone and know if I have simply missed something obvious.