Uploading large files from a Dexterity content type

Plone Support Customizing Plone
1

Here are my observations with my Plone setup:

Using Add File that came with Plone:
Small file: no issues
Large file (2.3 MB) no issues

Using a customized Dexterity content type with a File field to upload a file:
Small file: no issues
Large file (2.3 MB) the following errors:

2019-01-08 11:58:51 INFO ZEO.ClientStorage zeostorage Disconnected from storage: "('10.55.108.28', 8100)"
2019-01-08 11:58:51 INFO ZEO.asyncio.base Connected Protocol(('10.55.108.28', 8100), '1', False)
2019-01-08 11:58:51 INFO ZEO.ClientStorage zeostorage Reconnected to storage: ('10.55.108.28', 8100)
2019-01-08 11:58:51 ERROR Zope.SiteErrorLog 1546966731.860.674084955887 https://somesite.com/department/policies/section/test-1/@@edit
Traceback (innermost last):
Module ZPublisher.Publish, line 146, in publish
Module Zope2.App.startup, line 303, in commit
Module transaction._manager, line 131, in commit
Module transaction._transaction, line 310, in commit
Module transaction._transaction, line 301, in commit
Module transaction._transaction, line 446, in _commitResources
Module transaction._transaction, line 420, in _commitResources
Module ZODB.Connection, line 493, in commit
Module ZODB.Connection, line 1061, in _commit_savepoint
Module ZODB.mvccadapter, line 165, in store
Module ZEO.ClientStorage, line 567, in store
Module ZEO.asyncio.client, line 774, in async
Module ZEO.asyncio.client, line 748, in call
Module ZEO.asyncio.client, line 756, in wait_for_result
Module concurrent.futures._base, line 429, in result
Module concurrent.futures._base, line 381, in __get_result
ClientDisconnected

Plone version:

Plone 5.1.4 (5114)
CMF 2.2.12
Zope 2.13.27
Python 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
PIL 3.4.2 (Pillow)

Question: Is there a file size limit for a Dexterity File field? If so, how do I increase it?

Any help or advice would be most appreciated. Thank you so much.

1 Like
2

Can we see the code/definition for the custom content types?

3

Here are the codes. The problem occurs if the file is bigger than 800KB. I thought perhaps it was a blobcache issue but deleting and restarting Plone with a new blobcache did not solve the issue for files larger than 800KB.

PM_Administrative_Letter:

<?xml version="1.0" encoding="UTF-8"?>
<object
    i18n:domain="ncdhhscontenttype.policiesmanuals"
    meta_type="Dexterity FTI"
    name="PM_Administrative_Letter"
    xmlns:i18n="http://xml.zope.org/namespaces/i18n">

  <!-- Basic properties -->
  <property
      i18n:translate=""
      name="title">Administrative Letter</property>
  <property
      i18n:translate=""
      name="description"></property>

  <property name="allow_discussion">False</property>
  <property name="factory">PM_Administrative_Letter</property>
  <property name="icon_expr"></property>
  <property name="link_target">string:${portal_url}/document_icon.png</property>

  <!-- Hierarchy control -->
  <property name="allowed_content_types">
    <!-- <element value="" />-->
  </property>
  <property name="filter_content_types">True</property>
  <property name="global_allow">True</property>

  <!-- Schema, class and security -->
  <property name="add_permission">cmf.AddPortalContent</property>
  <property name="klass">plone.dexterity.content.Item</property>
  <property name="model_file">ncdhhscontenttype.policiesmanuals:models/pm_administrative_letter2.xml</property>
  <property name="model_source"></property>
  <!--property name="schema">ncdhhscontenttype.policiesmanuals.interfaces.IPM_Administrative_Letter</property-->
  <property name="schema"></property>

  <!-- Enabled behaviors -->
  <property name="behaviors">
    <element value="plone.app.content.interfaces.INameFromTitle" />
    <element value="plone.app.dexterity.behaviors.metadata.IDublinCore"/>
    <element value="collective.dexteritytextindexer.behavior.IDexterityTextIndexer"/>
    <element value="plone.app.dexterity.behaviors.exclfromnav.IExcludeFromNavigation"/>
    <element value="plone.app.contenttypes.behaviors.tableofcontents.ITableOfContents"/>
    <element value="plone.app.versioningbehavior.behaviors.IVersionable"/>
    <element value="plone.app.lockingbehavior.behaviors.ILocking"/>
  </property>

  <!-- View information -->
  <property name="add_view_expr">string:${folder_url}/++add++PM_Administrative_Letter</property>
  <property name="immediate_view">pm_administrativeletterview</property>
  <property name="default_view">pm_administrativeletterview</property>
  <property name="default_view_fallback">False</property>
  <property name="view_methods">
    <element value="view" />
  </property>

  <!-- Method aliases -->
  <alias
      from="(Default)"
      to="(dynamic view)"
  />
  <alias
      from="edit"
      to="@@edit"
  />
  <alias
      from="sharing"
      to="@@sharing"
  />
  <alias
      from="view"
      to="(selected layout)"
  />

  <!-- Actions -->
  <action
      action_id="view"
      category="object"
      condition_expr=""
      description=""
      i18n:attributes="title description"
      title="View"
      url_expr="string:${object_url}"
      visible="True">
    <permission value="View" />
  </action>
  <action
      action_id="edit"
      category="object"
      condition_expr=""
      description=""
      i18n:attributes="title description"
      title="Edit"
      url_expr="string:${object_url}/edit"
      visible="True">
    <permission value="Modify portal content" />
  </action>

</object>

pm_administrative_letter.xml:

<?xml version='1.0' encoding='utf8'?>
  <model 
    xmlns:form="http://namespaces.plone.org/supermodel/form" 
    xmlns:i18n="http://xml.zope.org/namespaces/i18n" 
    xmlns:indexer="http://namespaces.plone.org/supermodel/indexer" 
    xmlns:lingua="http://namespaces.plone.org/supermodel/lingua" 
    xmlns:marshal="http://namespaces.plone.org/supermodel/marshal" 
    xmlns:security="http://namespaces.plone.org/supermodel/security" 
    xmlns:users="http://namespaces.plone.org/supermodel/users" 
    xmlns="http://namespaces.plone.org/supermodel/schema">
    <schema>

      <field name="adminletter_number" type="zope.schema.TextLine" indexer:searchable="true">
        <description/>
        <title>Administrative Letter Number</title>
      </field>

      <field name="agency_division" type="zope.schema.Choice" indexer:searchable="true">
        <description/>
        <title>Agency/Division</title>
        <values>
          <element>Select one:</element>
          <element>NC Department of Health and Human Services (NCDHHS)</element>
          <element>-----------------------------------------</element>
          <element>Aging and Adult Services (DAAS)</element>
          <element>Child Development and Early Education (DCDEE)</element>
          <element>Controller, Office of the (OOC)</element>
          <element>Health Benefits/NC Medicaid (DHB)</element>
          <element>Health Service Regulation (DHSR)</element>
          <element>Mental Health, Developmental Disabilities, and Substance Abuse Services (DMHDDSAS)</element>
          <element>Public Health (DPH)</element>
          <element>Rural Health (ORH)</element>
          <element>Services for the Blind (DSB)</element>
          <element>Services for the Deaf and Hard of Hearing (DSDHH)</element>
          <element>Social Services (DSS)</element>
          <element>State Operated Healthcare Facilities (DSOHF)</element>
          <element>Vocational Rehabilitation Services (DVRS)</element>
        </values>
      </field>

      <field name="policy_program" type="zope.schema.Choice" indexer:searchable="true">
        <description/>
        <required>False</required>
        <title>Policy Program</title>
        <values>
          <element>Select one:</element>
          <element>Child Support</element>
          <element>Child Welfare</element>
          <element>Energy</element>
          <element>Food and Nutrition Services (FNS)</element>
          <element>Health Service Regulation</element>
          <element>Medicaid Family and Children</element>
          <element>Medicaid Aged, Blind, Disabled</element>
          <element>Medicaid Modified Adjusted Gross Income (MAGI)</element>
          <element>Refugee</element>
          <element>Special Assistance</element>
          <element>Special Assistance In-Home</element>
          <element>Subsidized Child Care Assistance</element>
          <element>Work First</element>
        </values>
      </field>

      <field name="adminletter_attention" type="zope.schema.Text" indexer:searchable="true">
        <description/>
        <required>False</required>
        <title>Attention</title>
      </field>

      <field name="adminletter_distribution" type="zope.schema.Text" indexer:searchable="true">
        <description/>
        <required>False</required>
        <title>Distribution</title>
      </field>

      <field name="adminletter_effective_date" type="zope.schema.Datetime" indexer:searchable="true">
        <description/>
        <title>Effective Date</title>
      </field>

      <field name="adminletter_content" type="plone.app.textfield.RichText" indexer:searchable="true">
        <description/>
        <required>False</required>
        <title>Administrative Letter Content</title>
      </field>

      <field name="adminletter_file" type="plone.namedfile.field.NamedBlobFile" indexer:searchable="true">
        <description/>
        <required>False</required>
        <title>File</title>
      </field>

    </schema>
  </model>
4

We store files with hundreds of MB through custom file fields without issues.

5

Are you sure there is enough disk space on the server?

6

Kim, thanks for your question. I have checked our servers. We do have lots of space. Two things I am checking out now:

  1. The network connection between Plone and Zope, each residing on different servers. We are required to separate web applications from the database and place them on different servers. Our networking guys are looking into this now.
  2. The other issue is the physical memory of our servers. Each has 4GB. Is this sufficient or should this be increased?

Thanks so much!

7

nginx and apache have settings for timeout and file uploads. E.g. nginx has a default limit of 1MB on file uploads.

8

Philip, thank so much for the advice. My Apache settings for timeout and file uploads were increased but to no avail. Here is my current setup:
post_max_size = 8M
upload_max_filesize = 10M

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

There was once when I restarted the Apache server, I could upload a 1.7MB file the first time. However, when I tried to upload it again, the same disconnect error message appeared.

9

Something isn't consistent here... big uploads work when you're adding a standard File content item, but not in this custom content type. And: it works only once after restarting Apache but not subsequently.

10

That sounds fine, but of course you should be monitoring memory use at least when you are running into a problem like this.

11

So we increased both CPU and memory over the weekend. There is still a lot of inconsistency. I could upload a huge file the first time and then it bomb out subsequently. What I have noticed today when I ran each client individually, 8080 and 8081, the latter showed an error that is rather curious: WARNING trollius socket.send() raised exception.

Have you all seen this before? Is this a Plone thing or a Python thing. What could be happening here? What is WARNING trollius socket.send() raised exception. Is there a way to fix this?

2019-01-22 14:04:42 WARNING trollius socket.send() raised exception.
.
.
.
.
2019-01-22 14:04:42 WARNING trollius socket.send() raised exception.
2019-01-22 14:04:42 INFO ZEO.ClientStorage zeostorage Disconnected from storage: "('10.55.107.40', 8100)"
2019-01-22 14:04:42 ERROR trollius Exception in callback _call_connection_lost(Connecti...by peer'))
handle: <Handle _call_connection_lost(Connecti...by peer'))>
Traceback (most recent call last):
File "/var/plone/sharedservices/buildout-cache/eggs/trollius-2.1-py2.7.egg/trollius/events.py", line 136, in _run
self._callback(*self._args)
File "/var/plone/sharedservices/buildout-cache/eggs/trollius-2.1-py2.7.egg/trollius/selector_events.py", line 626, in _call_connection_lost
self._sock.close()
AttributeError: 'NoneType' object has no attribute 'close'
2019-01-22 14:04:42 INFO ZEO.asyncio.base Connected Protocol(('10.55.107.40', 8100), '1', False)
2019-01-22 14:04:42 INFO ZEO.ClientStorage zeostorage Reconnected to storage: ('10.55.107.40', 8100)
2019-01-22 14:04:42 ERROR Zope.SiteErrorLog 1548183882.580.726887674881 http://somewhere.state.nc.us:8081/PoliciesManuals/++add++PM_Administrative_Letter
Traceback (innermost last):
Module ZPublisher.Publish, line 146, in publish
Module Zope2.App.startup, line 303, in commit
Module transaction._manager, line 131, in commit
Module transaction._transaction, line 310, in commit
Module transaction._transaction, line 301, in commit
Module transaction._transaction, line 446, in _commitResources
Module transaction._transaction, line 420, in _commitResources
Module ZODB.Connection, line 503, in commit
Module ZEO.ClientStorage, line 573, in checkCurrentSerialInTransaction
Module ZEO.asyncio.client, line 774, in async
Module ZEO.asyncio.client, line 748, in call
Module ZEO.asyncio.client, line 756, in wait_for_result
Module concurrent.futures._base, line 429, in result
Module concurrent.futures._base, line 381, in __get_result
ClientDisconnected

12

Is there a way for me to increase the size of my blobcache? Here is my server setup. Server1 one houses Plone and Server2 houses the database:

Server 1 (Web Server where Plone resides)

[zeoserver]
<= zeoserver_base
recipe = plone.recipe.zeoserver
zeo-address = Server2 IP address:8100

[client1]
<= client_base
recipe = plone.recipe.zope2instance
zeo-address = ${zeoserver:zeo-address}
http-address = 8080
zeo-client = on
# blobcache appears on client server and not zeoserver
shared-blob = off
blob-storage = ${:var}/blobcache

[client2]
<= client_base
recipe = plone.recipe.zope2instance
zeo-address = ${zeoserver:zeo-address}
http-address = 8081
zeo-client = on
# blobcache appears on client server and not zeoserver
shared-blob = off
blob-storage = ${:var}/blobcache

Server 2 (Data Server where ZOPE DB resides)

[zeoserver]
<= zeoserver_base
recipe = plone.recipe.zeoserver
zeo-address = 0.0.0.0:8100

[client1]
<= client_base
recipe = plone.recipe.zope2instance
zeo-address = ${zeoserver:zeo-address}
http-address = 8080

[client2]
<= client_base
recipe = plone.recipe.zope2instance
zeo-address = ${zeoserver:zeo-address}
http-address = 8081

When the files are less than 1 MB, it goes to blobcache on Server1 and then onto blobstorage in Server 2. However when the files are larger than 1 MB, it seems to not even register in blobcache on Server1. It is like it is stuck there not knowing what to do, and then it disconnects. I have already increase the physical memory of my server to 8G. Is there a way for me to increase the blobcache size? Any advise would be most appreciated. Thank you so much.

13

I think you should turn on the sharing blob option and mount your blobstorage via nfs to the Clients

14

Jan, if I turn on shared-blob would the blobstorage be on my client's server? I am not supposed to store any data on the clients' server. I am required to separate the web application from the database on two separate servers.

15

I got same error. I checked zeoserver.log, I found below:

2022-10-19T12:35:10 (0.0.0.0:8100) disconnected during unlocked transaction
2022-10-19T12:35:10 Connected server protocol
2022-10-19T12:35:10 received handshake 'Z5'
2022-10-19T12:35:10 (0.0.0.0:8100) no current transaction: tpc_abort()
2022-10-19T12:35:20 Bad async request, 'storeBlobStart'
Traceback (most recent call last):
  File "/XXXXXXX/buildout-cache/eggs/ZEO-5.2.2-py3.6.egg/ZEO/asyncio/server.py", line 103, in message_received
    result = getattr(self.zeo_storage, name)(*args)
  File "/XXXXXXX/buildout-cache/eggs/ZEO-5.2.2-py3.6.egg/ZEO/StorageServer.py", line 460, in storeBlobStart
    assert self.blob_tempfile is None
AssertionError
2022-10-19T12:35:20 Bad async request, 'storeBlobStart'
Traceback (most recent call last):
  File "/XXXXXXX/buildout-cache/eggs/ZEO-5.2.2-py3.6.egg/ZEO/asyncio/server.py", line 103, in message_received
    result = getattr(self.zeo_storage, name)(*args)
  File "/XXXXXXX/buildout-cache/eggs/ZEO-5.2.2-py3.6.egg/ZEO/StorageServer.py", line 460, in storeBlobStart
    assert self.blob_tempfile is None
AssertionError
2022-10-19T12:35:20 (0.0.0.0:8100) disconnected during unlocked transaction
2022-10-19T12:35:20 Connected server protocol
2022-10-19T12:35:20 received handshake 'Z5'
2022-10-19T12:35:20 (0.0.0.0:8100) no current transaction: tpc_abort()

Is it same situation?

16

I made a Plone issue: ZEO blobfile bug on Plone DX with large files · Issue #3660 · plone/Products.CMFPlone · GitHub

17

I ran into something likely similar when initialy setting up our ZEO. In notes from 2018, I see

What is apparently missing on [datacenter] is the ability to read the .tmp- files created in the blobstorage filesystem.

and that we then worked around it by making the ZEO client at the datacenter read-only ???, following Nathan van Gheem's comments here: Blobstorage errors when implementing Zope Replication Services (ZRS) on Plone 4.3.2 - Stack Overflow

Edit: sorry, looks like I gave you wrong info. Checking on the remote instance, this is the client config:

<zodb_db main>
    # Main database
    cache-size 30000
# Blob-enabled ZEOStorage database
    <zeoclient>
      read-only false
      read-only-fallback false
      blob-dir /usr/local/Plone5_ZEO/zeocluster/var/blobstorage
      shared-blob-dir off
      server xxx.xxx.xxx.xxx:8100
      storage 1
      name zeostorage
      var /usr/local/Plone5_ZEO/zeocluster/parts/client1/var
      cache-size 50MB
      blob-cache-size 100MB
      client mdb_cache
    </zeoclient>
    mount-point /
</zodb_db>

Local clients:

<zodb_db main>
    # Main database
    cache-size 30000
# Blob-enabled ZEOStorage database
    <zeoclient>
      read-only false
      read-only-fallback false
      blob-dir /usr/local/Plone5_ZEO/zeocluster/var/blobstorage
      shared-blob-dir on
      server 10.0.10.2:8100
      storage 1
      name zeostorage
      var /usr/local/Plone5_ZEO/zeocluster/parts/client1/var
      cache-size 128MB
    </zeoclient>
    mount-point /
</zodb_db>
18

My problem was fixed as ZEO == 5.3.

2 Likes
19

Thank you so much!!!