Another quirk (or at least opaqueness) of the api.create ... the saga continues

I swear, when I get this code working, I will post it on GitHub and post a note on this forum so that others who follow can learn from my frustration. I will also try to provide tweaks to the Plone API documentation to make it easier for others as well. Here goes round 3... (Note, the good news is that rounds 1 and 2 were successful, so I'm hopeful. Thanks to all who helped.)

Here is my situation...

I have a custom Dexterity object with a field called source_file which is a file property. (In this case, I took a Document object, cloned it and added a file property to it called source_file). The idea was to store a .pdf file in that field (although on some occasions, the file may be word-processor native docx or .wpd file instead. Yes, some people cranked out a lot of WordPerfect documents back in the day.

I'm trying to use the Plone API to load a file into the source_file property, and that file is a downloadable file on a server on the Internet, with url=url.

I'm able to download the file (seemingly correctly) with the following code...

import urllib

url = 'https://supreme.justia.com/cases/federal/us/531/98/case.pdf'
f = urllib.urlopen(url)

So far, so good...

When I try to upload the file (into the proper field (source_file)) via the create statement), e.g...

obj = api.content.create(type='opinion', ... source_file=f ...)

... This is what I got ...

$ sudo -u plone_daemon ./instance --object-path="/test" run ../case_api_test.py
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.add.DefaultAddView has a security declaration for nonexistent method 'browserDefault'
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.add.DefaultAddView has a security declaration for nonexistent method 'publishTraverse'
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.edit.DefaultEditView has a security declaration for nonexistent method 'browserDefault'
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.edit.DefaultEditView has a security declaration for nonexistent method 'publishTraverse'
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.delete.DefaultDeleteView has a security declaration for nonexistent method 'browserDefault'
2018-10-08 12:55:22 WARNING Init Class plone.app.tiles.browser.delete.DefaultDeleteView has a security declaration for nonexistent method 'publishTraverse'
2018-10-08 12:55:22 WARNING Init Class plone.app.drafts.storage.Storage has a security declaration for nonexistent method 'drafts'
2018-10-08 12:55:23 ERROR txn.140235036460800 Error in tpc_abort() on manager <Connection at 7f8af47fcb90>
Traceback (most recent call last):
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 461, in _cleanup
rm.tpc_abort(self)
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/Connection.py", line 631, in tpc_abort
transaction = transaction.data(self)
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 483, in data
raise KeyError(ob)
KeyError: <Connection at 7f8af47fcb90>
Traceback (most recent call last):
File "/opt/plone/zinstance/parts/instance/bin/interpreter", line 293, in
exec(_val)
File "", line 1, in
File "../case_api_test.py", line 26, in
obj = api.content.create(type='opinion', title=u'Bush v. Gore, 531 U.S. 90 (2000)', text=S, metadata=T, allow_discussion=True, subject=subject, source_file=f, container=portal)
File "", line 2, in create
File "/opt/plone/buildout-cache/eggs/plone.api-1.8.3-py2.7.egg/plone/api/validation.py", line 77, in wrapped
return function(*args, **kwargs)
File "", line 2, in create
File "/opt/plone/buildout-cache/eggs/plone.api-1.8.3-py2.7.egg/plone/api/validation.py", line 149, in wrapped
return function(*args, **kwargs)
File "/opt/plone/buildout-cache/eggs/plone.api-1.8.3-py2.7.egg/plone/api/content.py", line 126, in create
transaction.savepoint(optimistic=True)
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_manager.py", line 147, in savepoint
return self.get().savepoint(optimistic)
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 228, in savepoint
self._saveAndRaiseCommitishError() # reraises!
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 344, in _saveAndRaiseCommitishError
reraise(t, v, tb)
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 225, in savepoint
savepoint = Savepoint(self, optimistic, *self._resources)
File "/opt/plone/buildout-cache/eggs/transaction-2.1.2-py2.7.egg/transaction/_transaction.py", line 713, in init
savepoint = savepoint()
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/Connection.py", line 1000, in savepoint
self._commit(None)
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/Connection.py", line 547, in _commit
self._store_objects(ObjectWriter(obj), transaction)
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/Connection.py", line 579, in _store_objects
p = writer.serialize(obj) # This calls getstate of obj
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/serialize.py", line 419, in serialize
return self._dump(meta, obj.getstate())
File "/opt/plone/buildout-cache/eggs/ZODB-5.3.0-py2.7.egg/ZODB/serialize.py", line 428, in _dump
self._p.dump(state)
TypeError: can't pickle _ssl._SSLSocket objects


Okay, I said. Clearly not the right way to go about it. The object returned from urlopen is, after all, a fileish object, not actually a file object in the true sense, right? On the next try, I tweaked the code such that...

url = 'https://supreme.justia.com/cases/federal/us/531/98/case.pdf'
f = urllib.urlopen(url)**.read()**  # added the .read()

obj = api.content.create(type='opinion', ... source_file=f ...)

... that script ran correctly (no errors). Going back to Plone, I found the object right where it should have been. Could view it (no Mosaic view, just regular Page view (so no custom fields). However, when I tried to edit the object in Plone, this is what I got...

We’re sorry, but there seems to be an error…

Here is the full error message:

Display traceback as text

Traceback (innermost last):

Module ZPublisher.Publish, line 138, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 48, in call_object
Module plone.z3cform.layout, line 63, in __call__
Module plone.z3cform.layout, line 57, in update
Module z3c.form.form, line 162, in render
Module zope.browserpage.viewpagetemplatefile, line 49, in __call__
Module zope.pagetemplate.pagetemplate, line 137, in pt_render
Module five.pt.engine, line 98, in __call__
Module z3c.pt.pagetemplate, line 163, in render
Module chameleon.zpt.template, line 261, in render
Module chameleon.template, line 171, in render
Module 5d32cc37611f0d6df8260fa4635b9bfd.py, line 91, in render
Module b3af3f21f3ffbf485171a8eaa881c6de.py, line 1826, in render_titlelessform
Module b3af3f21f3ffbf485171a8eaa881c6de.py, line 451, in render_fields
Module b3af3f21f3ffbf485171a8eaa881c6de.py, line 126, in render_widget_rendering
Module b3af3f21f3ffbf485171a8eaa881c6de.py, line 1069, in render_field
Module five.pt.expressions, line 161, in __call__
Module Products.Five.browser.metaconfigure, line 485, in __call__
Module zope.browserpage.viewpagetemplatefile, line 81, in __call__
Module zope.browserpage.viewpagetemplatefile, line 49, in __call__
Module zope.pagetemplate.pagetemplate, line 137, in pt_render
Module five.pt.engine, line 98, in __call__
Module z3c.pt.pagetemplate, line 163, in render
Module chameleon.zpt.template, line 261, in render
Module chameleon.template, line 171, in render
Module 50cef1e26e3a98783d88506b469b2329.py, line 610, in render
Module 50cef1e26e3a98783d88506b469b2329.py, line 481, in render_widget_wrapper
Module five.pt.expressions, line 161, in __call__
Module z3c.form.widget, line 154, in render
Module zope.browserpage.viewpagetemplatefile, line 49, in __call__
Module zope.pagetemplate.pagetemplate, line 137, in pt_render
Module five.pt.engine, line 98, in __call__
Module z3c.pt.pagetemplate, line 163, in render
Module chameleon.zpt.template, line 261, in render
Module chameleon.template, line 191, in render
Module chameleon.template, line 171, in render
Module 319702e1a54ab9991a6dabb8188fe443.py, line 142, in render
Module five.pt.expressions, line 154, in __call__
Module five.pt.expressions, line 126, in traverse
Module zope.traversing.adapters, line 142, in traversePathElement
__traceback_info__: (<NamedFileWidget 'form.widgets.source_file'>, 'file_content_type')
Module zope.traversing.adapters, line 56, in traverse
__traceback_info__: (<NamedFileWidget 'form.widgets.source_file'>, 'file_content_type', ())

LocationError: (<NamedFileWidget 'form.widgets.source_file'>, 'file_content_type')

  • Expression: "widget/@@ploneform-render-widget"
  • Filename: ... rm-3.0.4-py2.7.egg/plone/app/z3cform/templates/macros.pt
  • Location: (line 100: col 81)
  • Source: ... place="structure widget/@@ploneform-render-widget"/>
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  • Expression: "view/file_content_type"
  • Filename: ... 2.0.5-py2.7.egg/plone/formwidget/namedfile/file_input.pt
  • Location: (line 28: col 21)
  • Source: doc_type view/file_content_type;
    ^^^^^^^^^^^^^^^^^^^^^^
  • Arguments: repeat: {...} (0)
    context: <ImplicitAcquisitionWrapper bush-v-gore-531-u-s-90-2000-1 at 0x7f701e86a0f0>
    exists: True
    views: <ViewMapper - at 0x7f701a4918d0>
    modules: <TraversableModuleImporter - at 0x7f702660df50>
    args: <tuple - at 0x7f702ea3f050>
    nothing: <NoneType - at 0x55fb98eb94d0>
    target_language: <NoneType - at 0x55fb98eb94d0>
    allow_nochange: True
    default: <object - at 0x7f702ea28540>
    request: <instance - at 0x7f701c0ee560>
    wrapped_repeat: {...} (0)
    download_url: http://localhost:8080/test/bush-v-gore-531-u-s-90-...
    loop: {...} (0)
    template: <ViewPageTemplateFile - at 0x7f701ddbab50>
    action: nochange
    translate: <function translate at 0x7f701a4220c8>
    options: {...} (0)
    view: <NamedFileWidget source_file at 0x7f701b8413d0>

I'm sure that I'm missing something really subtle, but it sure seems elusive. Anyone have an idea of what the problem is?

Check this: https://docs.plone.org/develop/plone/forms/files.html

# Attention, this is the blob-file object itself,
# opposed to the field with the same class-name used above!
from plone.namedfile.file import NamedBlobFile

blob_data = NamedBlobFile(some_data_string, filename=u'video.mp4')

As richtext or images you need to use a specific instance of a class to set it. Then you can do

obj = api.content.create(type='opinion', ... source_file=blob_data)