How to properly remove broken persistence objects?

So ... Recently I've migrated a large plone site to python 3.
Overall the process went ok (at least was what it seems).
Everything is working except for one little thing.

When trying to create pages with internal links I get the this:

2020-08-18 19:56:31,117 ERROR   [ZODB.Connection:809][waitress] Couldn't load state for BTrees.OIBTree.OIBTree 0x356bdb
Traceback (most recent call last):
  File "/home/plone/python3/eggs/ZODB-5.5.1-py3.7.egg/ZODB/Connection.py", line 795, in setstate
    self._reader.setGhostState(obj, p)
  File "/home/plone/python3/eggs/ZODB-5.5.1-py3.7.egg/ZODB/serialize.py", line 633, in setGhostState
    state = self.getState(pickle)
  File "/home/plone/python3/eggs/ZODB-5.5.1-py3.7.egg/ZODB/serialize.py", line 626, in getState
    return unpickler.load()
  File "/home/plone/python3/eggs/ZODB-5.5.1-py3.7.egg/ZODB/serialize.py", line 483, in find_global
    return factory(conn, modulename, name)
  File "/home/plone/python3/eggs/Zope-4.1.3-py3.7.egg/Zope2/Startup/datatypes.py", line 286, in simpleClassFactory
    m = __import__(module, _globals, _globals, _silly)
ModuleNotFoundError: No module named 'Products.PloneFormGen'
2020-08-18 19:56:31,842 ERROR   [Zope.SiteErrorLog:251][waitress] 1597791391.7520330.1817711126485435 https://ufal.br/ufal/institucional/orgaos-de-apoio/assessoramento/pei/@@edit
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 156, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 338, in publish_module
  Module ZPublisher.WSGIPublisher, line 256, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 62, in call_object
  Module plone.z3cform.layout, line 63, in __call__
  Module plone.z3cform.layout, line 47, in update
  Module plone.dexterity.browser.edit, line 58, in update
  Module plone.z3cform.fieldsets.extensible, line 65, in update
  Module plone.z3cform.patch, line 30, in GroupForm_update
  Module z3c.form.group, line 145, in update
  Module plone.app.z3cform.csrf, line 22, in execute
  Module z3c.form.action, line 98, in execute
  Module z3c.form.button, line 315, in __call__
  Module z3c.form.button, line 170, in __call__
  Module plone.dexterity.browser.edit, line 30, in handleApply
  Module z3c.form.group, line 126, in applyChanges
  Module zope.event, line 32, in notify
  Module zope.component.event, line 27, in dispatch
  Module zope.component._api, line 124, in subscribers
  Module zope.interface.registry, line 442, in subscribers
  Module zope.interface.adapter, line 607, in subscribers
  Module zope.component.event, line 36, in objectEventNotify
  Module zope.component._api, line 124, in subscribers
  Module zope.interface.registry, line 442, in subscribers
  Module zope.interface.adapter, line 607, in subscribers
  Module z3c.relationfield.event, line 73, in updateRelations
  Module five.intid.intid, line 41, in getId
  Module zope.intid, line 104, in getId
  Module ZODB.Connection, line 795, in setstate
  Module ZODB.serialize, line 633, in setGhostState
  Module ZODB.serialize, line 626, in getState
  Module ZODB.serialize, line 483, in find_global
  Module Zope2.Startup.datatypes, line 286, in simpleClassFactory
ModuleNotFoundError: No module named 'Products.PloneFormGen'

It seems the I still have a left overs from PloneFormGen, so I started a debug console and did:

from ZODB.utils import p64
oid = 0x356bdb
oid = p64(oid)
obj = site._p_jar.get(oid)
for i in obj.items():
    if i[0].object.__module__.startswith('Products.PloneFormGen'):
        print(i[0].object)

That gave me a few of this:

<persistent broken Products.PloneFormGen.content.formMailerAdapter.FormMailerAdapter ...
<persistent broken Products.PloneFormGen.content.form.FormFolder ...
...

So the question is, how to get rid of those broken persistent objects??

In the meantime, I created a fake PloneFormGen package just to avoid the error above. I created several fake persistent classes like this:

Products/PloneFormGen/content/fields.py

import persistent

class FGDateField(persistent.Persistent):
    pass

class FGFieldsetEnd(persistent.Persistent):
    pass
...

Now using the same code in debug mode I get this:

<Products.PloneFormGen.content.formMailerAdapter.FormMailerAdapter object at 0x7f26a8133c18 oid 0x33af89 in <Connection at 7f26d03c8ba8>>
<Products.PloneFormGen.content.form.FormFolder object at 0x7f26a8133c88 oid 0x33af8a in <Connection at 7f26d03c8ba8>>
<Products.PloneFormGen.content.fields.FGStringField object at 0x7f26a8133cf8 oid 0x33af8b in <Connection at 7f26d03c8ba8>>

I'm not getting errors anymore, but I still want to get rid of those broken objects, how should I do that ??

The most natural approach (first to try) is: just delete them from the BTree, i.e. del obj([i[0]) in your loop, commit after the loop. Whether this is sufficient depends on what the tree is used for: deleting items from it may introduce inconsistencies which may need fixing as well. For example, if it is used as part of a "Folder" (specifically BTreeFolder2), then the folder has 3 interrelated attributes: _tree (the objects themselves), _mt_index (meta data for the objects) and _count (the number of items); changing any of the trees requires updating of the other attributes to avoid inconsistencies.

1 Like

For an instance of BTreeFolder2 one can use folder.manage_cleanup(). This rebuilds the whole tree, so it can take some time, but at least you'll know is it was broken or not.

1 Like

Ok, to me it doesn't seen to be a BTreeFolder2, but I could very wrong. The tree object type says it is a instance of BTrees.OIBTree.OIBTree.

So I made a copy of the site to another server and tried to use del as suggested by @dieter, but it doesn't seen to work... Looking at the docs for OIBTree I noticed there is a pop method so I used that in my loop. Strangely, I had to run the code several times to completely remove all references to PloneFormGen. After every trace was gone, I committed the changes and tried to reproduce the error and couldn't.

So far it seems to be working, now I have to do the same thing in the live server. If anything goes south i'll come back to report.

Thanks for the help.

All these would be inside PloneFormGen objects/containers.

So, if you delete the FormGen content (and rebuild the catalog) you should be rid of them

I've already did that before migrating to python 3. Somehow those references remained.

I know of one potential problem with the modification of a BTree in a loop resulting in an "object changed during iteration". Was that what you have observed? In this case, you remember what should get deleted in the loop and then do the actual deleting later in a separate loop over the list of deletions. Otherwise, the deletion should work - as long as you delete all broken objects contained in a persistent object in the same transaction.