Best practice documentation on ZODB Debugging

Plone Support
2

Appendix

Migrating a ZODB from py2 to py3

Since people often encounter issues with their ZODB after migrating here is a quick dive into migrating a ZODB from Python 2 to Python 3.

The migration is basically calling the script zodbupdate in py3 with the parameter --convert-py3.

$ ./bin/zodbupdate --convert-py3

You need to pass it the location of the database, the defaul-encoding (utf8) and a fallback-encoding (latin1) for items where decoding to utf8 fails.

Example:

$ ./bin/zodbupdate --convert-py3 --file=var/filestorage/Data.fs --encoding=utf8 --encoding-fallback latin1

Updating magic marker for var/filestorage/Data.fs
Ignoring index for /Users/pbauer/workspace/projectx/var/filestorage/Data.fs
Loaded 2 decode rules from AccessControl:decodes
Loaded 12 decode rules from OFS:decodes
[...]
Committing changes (#1).

After that you should be able to use your ZODB in Python 3.

The process in a nutshell:

#. First, run bin/zodbupdate -f var/filestorage/Data.fs
So no python3 convert stuff yet!
This will detect and apply several explicit and implicit rename rules.

#. Then run bin/instance zodbverify.
If this still gives warnings or exceptions, you may need to define more rules and apply them with zodbupdate.
But you can still choose to migrate to py3 if this shows errors.

#. Using Python 3 run bin/zodbupdate --convert-py3 --file=var/filestorage/Data.fs --encoding utf8

#. For good measure, on Python 3 run bin/instance zodbverify.

Read the docs: https://docs.plone.org/manage/upgrading/version_specific_migration/upgrade_zodb_to_python3.html

See also: Zodbverify: Porting Plone with ZopeDB to Python3

Dealing with oids

Transforming oids from int to hex and text and vice versa:

>>> from ZODB.utils import p64
>>> oid = 0x2c0ab6
>>> p64(oid)
b'\x00\x00\x00\x00\x00,\n\xb6'

>>> from ZODB.utils import oid_repr
>>> oid = b'\x00\x00\x00\x00\x00,\n\xb6'
>>> oid_repr(oid)
'0x2c0ab6'

>>> from ZODB.utils import repr_to_oid
>>> oid = '0x2c0ab6'
>>> repr_to_oid(oid)
b'\x00\x00\x00\x00\x00,\n\xb6'

Get a path for blobs:

from ZODB.blob import BushyLayout
if isinstance(oid, int):
    # e.g. oid = 0x2c0ab6
    from ZODB.utils import p64
    oid = p64(oid)
return BushyLayout.oid_to_path(None, oid)

Load a obj by oid from ZODB in a pdb-prompt:

oid = 0x2c0ab6
from ZODB.utils import p64
app = self.context.__parent__
obj = app._p_jar.get(p64(oid))

Links

Quite a lot of people from the Plone/Zope communities have wriotten about this issue. I learned a lot from these posts:

10 Likes