Extremely Slow to Change Workflow State

I'm using plone 5. Specifially 5.2.2 but I think this has been an issue for a few versions now.

I have imported lots of data. on one of my smaller sites I have 80k objects in total, and under a specific folder have 3k objects. When the import completes, the folders are not in a published state.

When I select "Publish" to publish this folder, I have to wait a significant amount of time before plone returns. Even after going into advanced settings and making sure that 'Include contained items' is unchecked.

a folder with 4,229 objects: 58.8 seconds

I would assume this would be a fast transaction - just one object is having a workflow transition. But it takes forever.

Anyone also experiencing this? Any suggestions on how to speed this up?

I noticed reindexObjectSecurity being called on every subobject.

<FSControllerPythonScript at /RFA/english/multimedia/content_status_modify>
Line 50
Module Products.CMFCore.WorkflowTool, line 252, in doActionFor
Module Products.CMFCore.WorkflowTool, line 544, in _invokeWithNotification
Module Products.CMFCore.WorkflowTool, line 610, in _reindexWorkflowVariables
Module Products.CMFCore.CMFCatalogAware, line 116, in reindexObjectSecurity
Module Products.ZCatalog.CatalogBrains, line 64, in _unrestrictedGetObject

Changing the visibility of a folder may affect the visibility of its descendants. To avoid surprises, it is therefore possible that a reindexObjectSecurity (or similarly spelled) is routinely called. This is recursively called on all descendants.

To understand what happens here, I would use profiling. If there is not yet a solution integrated with Zope, you can change the workflow in an interactive session (--> bin/{instance|client1} debug) and use Python's cProfile and pstats modules there.

1 Like

You used to be able to delay indexing with collective.indexing but I think that functionality was added to Plone some versions ago. It would be great to come up with a smarter way of reindexing that didn’t touch contained items except when necessary.

CastleCMS sends long running operations like that to a celery queue. That would be a nice feature to get into Plone.

There is an ancient extension called Products.QueueCatalog (or similar) which allows to perform operation for some indexes offline. Not integrated into Plone, but it should be usable with not too much effort.

For the record, you might find this link useful. It mentions Products.QueueCatalog and Products.PloneQueueCatalog:

I have meanwhile published dm.zope.profile on PyPI. It should help you to understand what the time is used for.

Plone Foundation Code of Conduct