Can't pickle objects saving Dexterity Document in Products.QuillsEnabled (Plone 4)

Juron · February 28, 2024, 8:13pm

Hi Plone,

Moving Plone 4.3.20 sites to Dexterity, I want to save a Dexterity Document in the blog addon Products.QuillsEnabled, which is Archetypes. I get the following TypeError: Can't pickle objects in acquisition wrappers (see below)

Traceback (innermost last):
  Module ZPublisher.Publish, line 138, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 48, in call_object
  Module Products.CMFPlone.FactoryTool, line 478, in __call__
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 48, in call_object
  Module plone.z3cform.layout, line 66, in __call__
  Module plone.z3cform.layout, line 50, in update
  Module plone.dexterity.browser.edit, line 64, in update
  Module plone.z3cform.fieldsets.extensible, line 59, in update
  Module plone.z3cform.patch, line 30, in GroupForm_update
  Module z3c.form.group, line 145, in update
  Module plone.app.z3cform.csrf, line 21, in execute
  Module z3c.form.action, line 98, in execute
  Module z3c.form.button, line 315, in __call__
  Module z3c.form.button, line 170, in __call__
  Module plone.dexterity.browser.edit, line 28, in handleApply
  Module z3c.form.group, line 126, in applyChanges
  Module zope.event, line 31, in notify
  Module zope.component.event, line 24, in dispatch
  Module zope.component._api, line 136, in subscribers
  Module zope.component.registry, line 321, in subscribers
  Module zope.interface.adapter, line 585, in subscribers
  Module zope.component.event, line 32, in objectEventNotify
  Module zope.component._api, line 136, in subscribers
  Module zope.component.registry, line 321, in subscribers
  Module zope.interface.adapter, line 585, in subscribers
  Module plone.app.versioningbehavior.subscribers, line 62, in create_version_on_save
  Module Products.CMFEditions.CopyModifyMergeRepositoryTool, line 299, in save
  Module transaction._manager, line 101, in savepoint
  Module transaction._transaction, line 260, in savepoint
  Module transaction._transaction, line 257, in savepoint
  Module transaction._transaction, line 690, in __init__
  Module ZODB.Connection, line 1125, in savepoint
  Module ZODB.Connection, line 623, in _commit
  Module ZODB.Connection, line 658, in _store_objects
  Module ZODB.serialize, line 422, in serialize
TypeError: Can't pickle objects in acquisition wrappers.

How can I resolve this?

TYIA,
J.

dieter · February 28, 2024, 9:16pm

Juron via Plone Community wrote at 2024-2-28 20:23 +0000:

Hi Plone,

Moving Plone 4.3.20 sites to Dexterity, I want to save a Dexterity Document in the blog addon Products.QuillsEnabled, which is Archetypes. I get the following TypeError: Can't pickle objects in acquisition wrappers (see below)
Traceback (innermost last):
...
 Module plone.dexterity.browser.edit, line 28, in handleApply
 Module z3c.form.group, line 126, in applyChanges
 Module zope.event, line 31, in notify
 Module zope.component.event, line 24, in dispatch
...
 Module plone.app.versioningbehavior.subscribers, line 62, in create_version_on_save
 Module Products.CMFEditions.CopyModifyMergeRepositoryTool, line 299, in save
...
 Module ZODB.serialize, line 422, in serialize
TypeError: Can't pickle objects in acquisition wrappers.

An acquisition wrapper combines the base object with information
how this object has been accessed.
This access information is "runtime" information; it is very unlikely
that you would want to make it persistent.

There are two features which prevent that acquisition wrappers
are stored accidentally:

the above TypeError is raised when an attempt is made to
serialize (aka "pickle") an acquisition wrapper
when you assign an acquisition wrapper as attribute to a
persistent object, only the base object (i.e. without
the access information) is assigned.
usually ensures that you usually do not see the exception from 1.

In your case, an acquisition wrapper became part of a persistent
object. This should not have happened. It indicates a bug in the application.
To solve the problem, the bug must be located and fixed.
A first step is to identify the object and the acquisition wrapper.
You then check where (in the code) this object was modified and
the bug introduced.

For tasks of this kind, I very much appreciate Products.PDBDebugMode.
This extension enters the Python debugger in case of an exception
or error (provided your Plone runs in "development/debug" mode);
you can then use the debugger to analyse the state near
the error location -- e.g. for the exception above
find out which object should get serialized
and what of its attributes contains an acquisition wrapper.

Juron · February 28, 2024, 10:47pm

Thank you for your reply, @dieter.

I am not sure whether I am using Products.PDBDebugMode the right way. I installed it with the buildout, and then ran ./bin/client1 fg. I get a lot of output the moment I enter a title for the document in the blog. From there on, the site freezes until I enter "exit" on the debug console. This is the output I get: I do not see anything about an acquisition wrapper.

Pdb) 2024-02-28 23:28:21 ERROR root Exception while rendering an error message
Traceback (most recent call last):
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/SimpleItem.py", line 242, in raise_standardErrorMessage
    v = s(**kwargs)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Products.CMFCore-2.2.13-py2.7.egg/Products/CMFCore/FSPythonScript.py", line 127, in __call__
    return Script.__call__(self, *args, **kw)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Shared/DC/Scripts/Bindings.py", line 322, in __call__
    return self._bindAndExec(args, kw, None)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Shared/DC/Scripts/Bindings.py", line 359, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Products.PythonScripts-3.0-py2.7.egg/Products/PythonScripts/PythonScript.py", line 348, in _exec
    result = f(*args, **kw)
  File "Script (Python)", line 35, in standard_error_message
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Shared/DC/Scripts/Bindings.py", line 322, in __call__
    return self._bindAndExec(args, kw, None)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Shared/DC/Scripts/Bindings.py", line 359, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Products.CMFCore-2.2.13-py2.7.egg/Products/CMFCore/FSPageTemplate.py", line 237, in _exec
    result = self.pt_render(extra_context=bound_names)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Products.CMFCore-2.2.13-py2.7.egg/Products/CMFCore/FSPageTemplate.py", line 177, in pt_render
    self, source, extra_context
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Products/PageTemplates/PageTemplate.py", line 87, in pt_render
    showtal=showtal)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.pagetemplate-3.6.3-py2.7.egg/zope/pagetemplate/pagetemplate.py", line 132, in pt_render
    strictinsert=0, sourceAnnotations=sourceAnnotations
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.pagetemplate-3.6.3-py2.7.egg/zope/pagetemplate/pagetemplate.py", line 240, in __call__
    interpreter()
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 271, in __call__
    self.interpret(self.program)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 343, in interpret
    handlers[opcode](self, args)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tal-3.5.2-py2.7.egg/zope/tal/talinterpreter.py", line 867, in do_useMacro
    macro = self.engine.evaluateMacro(macroExpr)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/tales.py", line 696, in evaluate
    return expression(self)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/expressions.py", line 217, in __call__
    return self._eval(econtext)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Products/PageTemplates/Expressions.py", line 147, in _eval
    ob = self._subexprs[-1](econtext)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/zope.tales-3.5.3-py2.7.egg/zope/tales/expressions.py", line 124, in _eval
    ob = self._traverser(ob, element, econtext)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/Products/PageTemplates/Expressions.py", line 74, in boboAwareZopeTraverse
    object = object.restrictedTraverse(name)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/Traversable.py", line 317, in restrictedTraverse
    return self.unrestrictedTraverse(path, default, restricted=True)
  File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/Traversable.py", line 269, in unrestrictedTraverse
    obj, obj, None, next):
Unauthorized: You are not allowed to access 'main_template' in this context
> /usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/Traversable.py(269)unrestrictedTraverse()
-> obj, obj, None, next):

dieter · February 29, 2024, 6:39am

Juron via Plone Community wrote at 2024-2-28 22:58 +0000:

...
I am not sure whether I am using Products.PDBDebugMode the right way.
...
I get a lot of output the moment I enter a title for the document in the blog. From there on, the site freezes until I enter "exit" on the debug console. This is the output I get: I do not see anything about an acquisition wrapper.
Pdb) 2024-02-28 23:28:21 ERROR root Exception while rendering an error message

The Pdb) shows you that you have entered the debugger.
This happens because of the ERROR log entry:
Products.PDBDebugMode enters the debugger when an exception
reaches the framework (i.e. is not (completely) handled by the application)
of an error is logged.

The site apparently freezes because the debugger waits for commands.
Enter the continue (can be abbreviated as c) debugger command
to proceed.

Traceback (most recent call last):
File "/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/SimpleItem.py", line 242, in raise_standardErrorMessage

It might be necessary to fix the "error message" problem first,
to easily analyze the "aquisition wrapper" problem.

While I consider Products.PDBDebugMode an extremely useful tool,
I note that there are also associated problems.

The most important one (from my point of view):
Plone/Zope is a multithread system; this means that exceptions/errors
can happen in several threads almost at the same time.
In this case, the debugger is entered in several threads at the same
time and you do not really know to which debugger instance your
commands are sent.
This is not (yet) your problem.

The second problem is that the debugger is entered before
the problem you want to analyse. This is what you see
currently. If this happens only rarely, you can
enter the debugger command Continue to proceed to the next
exception/error. Otherwise, you must fix or work around the
preceding errors/exceptions.

In hard cases (when Products.PDBDebugMode cannot help me),
I use instead source code breakpoints: i.e. I modify the source
introducing import pdb; pdb.set_trace(). At those places,
the debugger is entered.
Note that this has the same problem as Products.PDBDebugMode
potentially entering the debugger in several threads at the same time.
You might need to put the set_trace inside a conditional statement
to reduce this risk.

Unauthorized: You are not allowed to access 'main_template' in this context

/usr/home/dev/plone.4.3.20/buildout-cache/eggs/Zope2-2.13.30-py2.7.egg/OFS/Traversable.py(269)unrestrictedTraverse()

I remember that this problem has been discussed recently
in a separate thread: apparently, Plone's error message
requires quite high priviledges (a Plone design bug).

To work around: temporarily register your own exception view
with a minimal template (accessing only information publicly available).
Details in the other thread.

Juron · February 29, 2024, 12:28pm

Thank you again, @dieter.

The Unauthorized: You are not allowed to access-error when I enter a title for the document in the blog is gone when I first publish the folder where Quills is in (make it visible for anonymous). By the way, I am logged in as admin.

Now when I save a document as entry in Quills, the pdb) shows the two lines below extra compared to the traceback in the initial question.

> /usr/home/dev/plone.4.3.20/buildout-cache/eggs/ZODB3-3.10.7-py2.7-freebsd-13.2-RELEASE-p10-amd64.egg/ZODB/serialize.py(422)serialize()
-> return self._dump(meta, obj.__getstate__())

Strange thing is that I can save and publish a document in the folder that is activated as Quillsblog, but I get the pickle-issue when I add a document by way of the Add Entry option of QuillsEnabled.

dieter · February 29, 2024, 3:00pm

Juron via Plone Community wrote at 2024-2-29 12:38 +0000:

...
Now when I save a document as entry in Quills, the pdb) shows the two lines below extra compared to the traceback in the initial question.
> /usr/home/dev/plone.4.3.20/buildout-cache/eggs/ZODB3-3.10.7-py2.7-freebsd-13.2-RELEASE-p10-amd64.egg/ZODB/serialize.py(422)serialize()
-> return self._dump(meta, obj.__getstate__())

You now use debugger commands to analyse the state,
e.g. p obj will "print obj",
p obj.__class__ will "print its class".

The problematic acquisition wrapper is likely somewhere in
the object's state, obtained via obj.__getstate__().
Unfortunately, they are not easy to recognize: an acquisition wrapper
tries hard to behave like the base object -- you do not easily see
a difference.

One way to recognize an acquisition wrapper is to call type on it.
This will reveal a wrapper reliably.

Thus, you must quess where in the object's state an acquisition wrapper
might be hidden. The wrapper will not be a top level attribute;
look for objects contained in top level objects, e.g. list elements,
attributes of (non persistent) subobjects, ...
You then call type on the guessed object to verify that it is indeed
an acquisition wrapper.

Strange thing is that I can save and publish a document in the folder that is activated as Quillsblog, but I get the pickle-issue when I add a document by way of the Add Entry option of QuillsEnabled.

Likely, the latter does something wrong.

Juron · February 29, 2024, 9:27pm

I used the following print commands in the debugger. Other commands I tried were incorrect and resulted in a SyntaxError.

(Pdb) p obj.__class__
<class 'Products.CMFPlone.FactoryTool.TempFolder'>

(Pdb) p obj
<TempFolder at /Plone/en/blogfolder/portal_factory/Document>

(Pdb) p obj.__getstate__
<built-in method __getstate__ of Acquisition.ImplicitAcquisitionWrapper object at 0x9adca3a00>

(Pdb) p self._dump
<bound method ObjectWriter._dump of <ZODB.serialize.ObjectWriter instance at 0x9a5eefd20>

(Pdb) p self._dump(meta, obj.__getstate__())
*** TypeError: TypeError("Can't pickle objects in acquisition wrappers.",)

I understand too little about this to know what it means and what I need to do next.

I am under the impression that the issue has to do with the fact that in the code of Products.QuillsEnabled there are references to ATDocument. For example, in configure.zcml:

  <adapter
      for="Products.ATContentTypes.interface.IATDocument"
      provides="quills.core.interfaces.IWeblogEntry"
      factory=".adapters.document.Document2WeblogEntry" />

And:

  <class class="Products.ATContentTypes.content.document.ATDocument">
       <implements interface="quills.core.interfaces.IPossibleWeblogEntry" />
  </class>

And in ./adapters/document.py:

class Document2WeblogEntry(Explicit, QuillsMixin):
    """Adapts an ATDocument to IWeblogEntry.

    >>> from zope.interface.verify import verifyClass
    >>> verifyClass(IWorkflowedWeblogEntry, Document2WeblogEntry)
    True
    """

For me, it seems like the problem arises because Quills expects an ATDocument, but instead is being fed with a Dexterity Document

riker11451 · March 1, 2024, 3:21pm

What are you trying to achieve here? Are you trying to port Quills to dexterity? This probably isn't a great idea, for both your sanity and productivity. Besides P4a add-ons, all the Plone blogging tools seemed to burn individuals (wicked).

Have you thought about exporting the content using say collective.exportimport, then creating either a new content type or modifying an existing one?

Juron · March 1, 2024, 7:18pm

Thank you for asking, @riker11451.

In fact, I try to achieve world dominance with Plone 4 Archetypes, but with Dexterity these days, I thought it is a good thing to move first to the new contenttype before the takeover.

No. After moving most of the things to Dexterity, I ran into an issue. Now I understand (I think) that it is because Quills does not accept that it gets a Dexterity contenttype to save. At the same time, I learned about debugging.

Yes, I have thought about export-import from an Archetype-oriented Plone 4 to Dexterity Plone 4. And I have thought about modifying existing code. I thought about making QuillsEnabled accept that it gets Dexterity contenttypes to save, but I do not know yet how.

QuillsEnabled is, for me, the best blogging tool on Plone. I know of no convincing competitor.

Edit: And I created a new content type. Quills cannot be activated on a Dexterity folder. So, I created an Archetype Blog Folder that can be activated for Quills.