The instances run after 34 days, and it use more and more memory... also, the site is becoming laggy.
Everything runs faster after restarting the instance.
Anyone have same issue?
We restart our clients weekly. It's not a Problem with Plone 6, its' Plone 
1 Like
Possibly a common behaviour due to various reasons, mostly related to Python. Regular restarts can be considered best practice if you find yourself in such a situation.
It's not a Plone problem, it's caused by using a dynamic language with a runtime that uses GC (Garbage Collection) to free objects in memory and can cause memory fragmentation. It's a well known problem with long running (> 7 days) Python processes that (de)-allocate memory for a lot of objects in memory. For large/busy sites you run multiple servers which you can do rolling restarts for every 2-7 days.
Python 2 had much more issues with this than Python3, but Python 3 can still show this behavior as well. Every design choice has advantages and drawbacks, this is one of the drawbacks.
It doesn't mean however that your specific codebase doesn't have a memory leak in its custom code. But one of the few ways to expose it is to monitor the memory usage over time and watch the graph. If it's a steep rise and then a slow curve to a 'limit' you're fine and this is the normal (de)-allocating of memory slowly going to a maximum. It's comparable to the concept of entropy in physics. If this maximum is bigger than your available server memory, lower the zodb-cache-size or zserver/wsgi threads.
If you see the same curve but then after a while a more straight line, you have a memory leak: per every requst or action somewhere some memory is lost and the used memory continues to grow linearly.
I've seen people increasing the allowed cache sizes and number of threads for Zope, then see Zope using all server memory rather quickly and then draw the conclusion "memory leak". The only way to really test this is to actually lower the threads/zodb-cache size so that after a restart you first see the normal elliptic curve but still have memory space left to observe the linearly increasing memory leak taking over after the normal growth stalls. THEN you have some heuristic proof.
8 Likes
Just out of curiosity: does anybody know about GitHub - bloomberg/memray: Memray is a memory profiler for Python and how it maybe could help to analyze memory consumptions within Plone?
This is because Python 3.11 introduced a change that causes Plone to leak memory.
There are two ways to reduce memory leaks in Plone 6 on Python 3.11:
- Use the Python garbage collector to collect unused objects. This will reduce the amount of memory that Plone uses, but will slow down your site.
- Upgrade to Python 3.12. This will fix the memory leak issue and will not slow down your site.
If you are not running Plone 6 on Python 3.11, you do not need to take any action.
1 Like
Can you point us to a Python issue or likewise to put some meat on the bones?
1 Like
Can you point us to a Python issue or likewise to put some meat on the bones?
Only references I could see related to async code which is not relevant to plone.
I however am seeing memory growing in our plone6 instances that looks different than plone5.
I might be a leak related to our customisations so we are still investigating.
An instance thats using 350MB of resident memory will start to increase in swap over time. So somethings being created and not used or gc.
There was this memory leak fixed in a recent Plone version: change interface declaration type #186 by mamico · Pull Request #187 · plone/plone.dexterity · GitHub
At least for me, which I manually backported it to Plone 5.2.x, it fixed our memory problems.
Check if you are using plone.dexterity==3.0.3 which comes with Plone==6.0.7 and then monitor again 
@gforcada awesome. will upgrade and report back.
For the record these are the kind of top ref counts I'm seeing. not sure if it gels with the interface declaration issue or not.
<select name="foo" size="10" class="form-control">
<option>plone.rest.traverse.RESTTraverse: 26381</option>
<option>plone.i18n.negotiate.negotiate.NegotiateLanguage: 26327</option>
<option>zope.interface.declarations.Implements: 17629</option>
<option>builtins.object: 15992</option>
<option>DateTime.DateTime.DateTime: 13770</option>
<option>plone.rest.traverse.MarkAsRESTTraverser: 13132</option>
<option>five.intid.keyreference.KeyReferenceToPersistent: 8663</option>
<option>_frozen_importlib_external.SourceFileLoader: 5107</option>
<option>z3c.relationfield.relation.RelationValue: 4492</option>
<option>_frozen_importlib.ModuleSpec: 4003</option>
<option>_frozen_importlib_external.ExtensionFileLoader: 3794</option>
<option>plone.restapi.serializer.dxfields.DefaultFieldSerializer: 3423</option>
<option>zope.interface.interface.Method: 2956</option>
<option>BTrees.Length.Length: 2796</option>
<option>zope.interface.declarations.ClassProvides: 2709</option>
<option>zope.configuration.xmlconfig.ParserInfo: 2518</option>
<option>builtins.type: 2433</option>
<option>pkg_resources.Requirement: 2349</option>
<option>plone.folder.default.DefaultOrdering: 2249</option>
<option>plone.locking.lockable.TTWLockable: 2171</option>
<option>plone.dexterity.behavior.DexterityBehaviorAssignable: 1997</option>
<option>Products.PageTemplates.engine.MappedExprType: 1858</option>
<option>Products.Five.browser.BrowserView: 1846</option>
<option>builtins.Exception: 1770</option>
<option>builtins.str: 1767</option>
<option>zope.publisher.browser.BrowserView: 1641</option>
<option>zope.location.location.Location: 1584</option>
<option>plone.restapi.serializer.dxfields.CollectionFieldSerializer: 1409</option>
<option>Products.MimetypesRegistry.MimeTypeItem.MimeTypeItem: 1355</option>
<option>builtins.BaseException: 1289</option>
<option>_frozen_importlib_external.FileFinder: 1250</option>
<option>_frozen_importlib_external.SourcelessFileLoader: 1246</option>
<option>builtins.PermissionRole: 1198</option>
<option>plone.app.contenttypes.content.Image: 1183</option>
<option>zope.traversing.namespace.view: 1110</option>
<option>Products.CMFCore.Expression.Expression: 999</option>
<option>plone.dexterity.browser.traversal.DexterityPublishTraverse: 977</option>
<option>builtins.int: 937</option>
<option>zope.interface.declarations.implementer: 880</option>
<option>persistent.Persistent: 874</option>
<option>plone.restapi.serializer.dxcontent.DexterityObjectPrimaryFieldTarget: 872</option>
<option>plone.dexterity.primary.PrimaryFieldInfo: 835</option>
<option>plone.namedfile.file.NamedBlobImage: 819</option>
<option>zope.interface.interface.Attribute: 772</option>
<option>plone.volto.content.FolderishDocument: 735</option>
<option>Products.Five.browser.metaconfigure.simple: 735</option>
<option>plone.app.contenttypes.content.File: 734</option>
<option>cffi.model.PointerType: 726</option>
<option>cusy.restapi.info.services.siteinfo.get.SiteInfo: 715</option>
<option>plone.restapi.services.actions.get.Actions: 713</option>
<option>plone.restapi.services.workflow.info.WorkflowInfo: 712</option>
<option>plone.restapi.services.types.get.TypesInfo: 712</option>
<option>plone.restapi.services.navroot.get.Navroot: 712</option>
<option>plone.restapi.services.contextnavigation.get.ContextNavigation: 712</option>
<option>plone.restapi.services.breadcrumbs.get.Breadcrumbs: 712</option>
<option>plone.restapi.services.aliases.get.Aliases: 712</option>
<option>cusy.restapi.info.services.contentinfo.get.ContentInfo: 712</option>
<option>collective.volto.formsupport.restapi.services.form_data.form_data.FormData: 712</option>
<option>collective.volto.blocksfooter.api.services.blocksfooter_settings.get.VoltoBlocksFooterSettings: 712</option>
<option>plone.restapi.services.navigation.get.Navigation: 711</option>
<option>plone.restapi.serializer.dxfields.ChoiceFieldSerializer: 709</option>
<option>plone.restapi.serializer.relationfield.RelationListFieldSerializer: 707</option>
<option>Products.ZCatalog.plan.Benchmark: 703</option>
<option>Products.Five.browser.pagetemplatefile.ViewPageTemplateFile: 697</option>
<option>builtins.bytes: 687</option>
<option>plone.volto.transforms.PreviewImageResolveUIDSerializer: 683</option>
<option>plone.volto.transforms.NestedResolveUIDSerializer: 683</option>
<option>plone.restapi.serializer.blocks.TextBlockSerializer: 683</option>
<option>plone.restapi.serializer.blocks.SlateTableBlockSerializer: 683</option>
<option>plone.restapi.serializer.blocks.SlateBlockSerializer: 683</option>
<option>plone.restapi.serializer.blocks.ResolveUIDSerializer: 683</option>
<option>collective.volto.formsupport.restapi.serializer.blocks.FormSerializerContents: 683</option>
<option>collective.volto.blocksfield.restapi.serializers.blocks.TextBlockSerializer: 683</option>
<option>collective.volto.blocksfield.restapi.serializers.blocks.ResolveUIDSerializer: 683</option>
<option>ExtensionClass.Base: 683</option>
<option>plone.restapi.services.Service: 678</option>
<option>plone.restapi.serializer.summary.SiteRootJSONSummarySerializer: 676</option>
<option>ZODB.blob.Blob: 644</option>
<option>plone.restapi.serializer.blocks.BlocksJSONFieldSerializer: 622</option>
<option>builtins.bool: 617</option>
<option>zope.component._declaration._adapts_descr: 550</option>
<option>builtins.tuple: 549</option>
<option>builtins.Acquirer: 545</option>
<option>Persistence.Persistent: 505</option>
<option>zope.schema.vocabulary.SimpleTerm: 490</option>
<option>abc.ABCMeta: 484</option>
<option>zope.schema._bootstrapfields.TextLine: 480</option>
<option>Products.Five.browser.metaconfigure.ViewMixinForTemplates: 474</option>
<option>OFS.SimpleItem.SimpleItem: 469</option>
<option>Products.CMFUid.UniqueIdAnnotationTool.UniqueIdAnnotation: 457</option>
<option>Products.CMFCore.WorkflowTool.DefaultWorkflowStatus: 442</option>
<option>plone.app.iterate.dexterity.policy.CheckinCheckoutPolicyAdapter: 419</option>
<option>cffi.model.FunctionPtrType: 418</option>
<option>Products.PageTemplates.Expressions.TrustedSubPathExpr: 406</option>
<option>plone.restapi.serializer.dxfields.ImageFieldSerializer: 395</option>
<option>zope.i18n.locales.LocaleCurrency: 391</option>
<option>zope.browserpage.metaconfigure.simple: 374</option>
<option>z3c.form.button.Button: 368</option>
<option>Products.PageTemplates.Expressions.TrustedZopePathExpr: 366</option>
<option>ExtensionClass.ExtensionClass: 365</option>
</select>
The second one (plone.i18n) it is mentioned on the memory leak discussion: Plone/ZCA memory leak? · Issue #3829 · plone/Products.CMFPlone · GitHub
The first one (plone.rest) not, but they might be hopefully 