Hello,
we are running a PLone 4.3 installation from 2014, with a huge and constantly increasing catalog (several hundred thousands items).
Searching is always a pain, so I'm looking for a way to increase its performance; we are wondering if setting an ExpriationDate for older items could help us: excluding them from searches will "lighten" the catalog?
Of course, we want to be able to recover them if needed (so, I think that unsetting the ExpirationDate will bring them to the original status).
Maybe it's a silly question, but despite reading the docs I can't really get the expected behaviour of an "Expired" item.
You could increase the number of objects cached in the ZODB cache. This would improve your searches, provided they are frequent (i.e. with a high chance that the affected index object components are used frequently and therefore cached).
Unfortunatety, it would increase your memory consumption. To get more control, some people put the catalog in a separate ZODB and increase the cache size only for this ZODB (the size of catalog objects has a much lower variance than that for general objects; thus, the effect of an increase of the cache size on memory consumption is more predictable than in the general case).
Thank you for your quick answer.
But setting the expiration date wouldn't help at all?
Our aim is to "archive" older items and in the meanwhile increase the overall searching performance.
(another way suggested by someone is to create a second contanier, put older items inside it, and recover them if needed, but I think its an overcomplex solution)
You may try it.
Searching has two important cost factors: the actual search and the sorting of the search result (if any). Setting an expiration date would decrease the size of the search result and thereby may reduce the sorting time. It is more difficult to assess the effect on the time for the actual search; maybe, it, too, will decrease.
Zope's search engine (the "ZCatalog") is not very efficient as it sits on top of a general purpose object oriented database (the "ZODB") designed for flexibility and ease of use and not for efficiency. If modifications are fairly rare (compared to reading uses), then replacing the "ZCatalog" by e.g. Sol/r will likely give you a much better search performances. Of course, such a replacement would be a huge step.