I am running 4 plone sites (5.2.4) served via 2 zeoclients. I have a bunch of monitoring scripts setup on the server to monitor memory, cpu usage, etc and do graceful restarts.
I have crons to restart plone every Sunday night.
Once restarted, each client uses 10% memory, no biggie. Every few hours each client goes up by about 1%. Usually by the end of the week, Saturday, I start get pinged about low server memory (80% in use).
Now, prior to this I had these exact 4 sites running 5.0.4 without this memory growth issue.
Hi David.
We moved a 2.5 million object system from 4.3 to 5.2 (python2 -> python3 also!) 18 months ago.
During process we learnt how our own python code impacts RAM usage
Could you add some more details related size of Data.fs, your python version, number of threads for each zeo client and maybe also number of objects.
Then maybe could give you a starting point for your own inspection of what impacts RAM usage.
Plone: Upgraded from 5.0.4 to 5.2.4
Python: 2.7 to 3.8
Data.fs: 967.1MB
Total Objects: 1,672,715
Zeoclients: 2
Threads: Whatever the plone default is.
Addons: A few simple products with only dexterity models, nothing special. EasyForm, PlonePAS.
All objects were freshly made, no moving the data.fs. A pipe was made to push out the data as json and create new objects in 5.2.4.
If you use a debug version of Python 3, sys._debugmallocstats gives you information about Python's memory management. This manages smaller memory blocks. The information allows you to estimate the memory fragmentation for those blocks.
If you are on Linux and use a fairly new glibc, then there is the "C" library function mallinfo2. You can call it from Python via ctypes. It provides information about the "C" memory management -- used for the management of large blocks. With it, you can check whether large blocks are leaked.
With older glibc version, there is only mallinfo. It provides the same information as mallinfo2 but uses only 4 bytes. If the memory management exceeds 4 GB, important information is truncated.