KeyError no local roles plugin

I have a strange bug on a dev site that is very difficult to reproduce, but I do have some questions on how to debug further, following the explanation. There are two things of note in the traceback. One we are using collective.elasticsearch, two this calls processQueue so this at least involves catalog optimization and is thus related in some way to a previous transaction.

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 156, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 338, in publish_module
  Module ZPublisher.WSGIPublisher, line 256, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 62, in call_object
  Module zope.browserpage.simpleviewclass, line 41, in __call__
  Module Products.Five.browser.pagetemplatefile, line 126, in __call__
  Module Products.Five.browser.pagetemplatefile, line 61, in __call__
  Module zope.pagetemplate.pagetemplate, line 135, in pt_render
  Module Products.PageTemplates.engine, line 88, in __call__
  Module z3c.pt.pagetemplate, line 173, in render
  Module chameleon.zpt.template, line 306, in render
  Module chameleon.template, line 209, in render
  Module chameleon.utils, line 75, in raise_with_traceback
  Module chameleon.template, line 187, in render
  Module 485cb277fa9b0bf6634f1ecb0a934611.py, line 4424, in render
  Module 4d9af5b1f80cd75806f779ed1236f777.py, line 581, in render_master
  Module z3c.pt.expressions, line 70, in render_content_provider
  Module zope.viewlet.manager, line 155, in update
  Module zope.viewlet.manager, line 161, in _updateViewlets
  Module plone.app.layout.viewlets.common, line 407, in update
  Module plone.memoize.view, line 48, in memogetter
  Module plone.app.layout.viewlets.common, line 403, in portal_tabs
  Module Products.CMFPlone.browser.navigation, line 146, in topLevelTabs
  Module collective.elasticsearch.patches, line 25, in safeSearchResults
  Module ims.elasticsearch.patches, line 52, in searchResults
  Module Products.CMFPlone.CatalogTool, line 429, in searchResults
  Module Products.CMFCore.indexing, line 96, in processQueue
  Module Products.CMFCore.indexing, line 223, in process
  Module Products.CMFCore.indexing, line 51, in reindex
  Module Products.CMFCore.CatalogTool, line 367, in _reindexObject
  Module collective.elasticsearch.patches, line 10, in catalog_object
  Module collective.elasticsearch.es, line 208, in catalog_object
  Module Products.CMFPlone.CatalogTool, line 351, in catalog_object
  Module Products.ZCatalog.ZCatalog, line 499, in catalog_object
  Module Products.ZCatalog.Catalog, line 369, in catalogObject
  Module Products.PluginIndexes.unindex, line 239, in index_object
  Module Products.PluginIndexes.KeywordIndex.KeywordIndex, line 60, in _index_object
  Module Products.PluginIndexes.KeywordIndex.KeywordIndex, line 95, in _get_object_keywords
  Module plone.indexer.wrapper, line 65, in __getattr__
  Module plone.indexer.delegate, line 20, in __call__
  Module Products.CMFPlone.CatalogTool, line 158, in allowedRolesAndUsers
  Module Products.PlonePAS.pas, line 401, in _getAllLocalRoles
  Module Products.PluginRegistry.PluginRegistry, line 108, in listPlugins
  Module Products.PluginRegistry.PluginRegistry, line 368, in _getPlugins
KeyError: <InterfaceClass Products.PlonePAS.interfaces.plugins.ILocalRolesPlugin>

The problem is completely transient. PluginRegistry raises a KeyError if it can't find any plugins for the given type, but I quite clearly have the Plone default borg_localroles. Indeed on a refresh the page loads without issue. What could make it unable to find the plugin temporarily??

The traceback here shows it is trying to get the local roles of some context to update that index in the catalog. This happens by calling processQueue() first which finishes any pending indexing as part of catalog optimization since 5.1. Is this supposed to (potentially) involve indexing from other requests? potentially stemming from other user actions? In some cases this appears to even trip CSRF protection because the request is now doing a db write.

I've had this happen on several different pages on the site, including edit form submissions, viewing a page, and even viewing a control panel form.

This was a reply via email (which did not read the forum):

Esoth via Plone Community wrote at 2020-3-5 16:49 +0000:
>I have a strange bug on a dev site that is very difficult to reproduce, but I do have some questions on how to debug further, following the explanation. There are two things of note in the traceback. One we are using collective.elasticsearch, two this calls processQueue so this at least involves catalog optimization and is thus related in some way to a previous transaction. 
>
>```
>Traceback (innermost last):
> ...
>  Module collective.elasticsearch.patches, line 25, in safeSearchResults
>  Module ims.elasticsearch.patches, line 52, in searchResults
>  Module Products.CMFPlone.CatalogTool, line 429, in searchResults
>  Module Products.CMFCore.indexing, line 96, in processQueue
>  Module Products.CMFCore.indexing, line 223, in process
>  Module Products.CMFCore.indexing, line 51, in reindex
>  Module Products.CMFCore.CatalogTool, line 367, in _reindexObject
>  Module collective.elasticsearch.patches, line 10, in catalog_object
>  Module collective.elasticsearch.es, line 208, in catalog_object
>  Module Products.CMFPlone.CatalogTool, line 351, in catalog_object
>  Module Products.ZCatalog.ZCatalog, line 499, in catalog_object
>  Module Products.ZCatalog.Catalog, line 369, in catalogObject
>  Module Products.PluginIndexes.unindex, line 239, in index_object
>  Module Products.PluginIndexes.KeywordIndex.KeywordIndex, line 60, in _index_object
>  Module Products.PluginIndexes.KeywordIndex.KeywordIndex, line 95, in _get_object_keywords
>  Module plone.indexer.wrapper, line 65, in __getattr__
>  Module plone.indexer.delegate, line 20, in __call__
>  Module Products.CMFPlone.CatalogTool, line 158, in allowedRolesAndUsers
>  Module Products.PlonePAS.pas, line 401, in _getAllLocalRoles
>  Module Products.PluginRegistry.PluginRegistry, line 108, in listPlugins
>  Module Products.PluginRegistry.PluginRegistry, line 368, in _getPlugins
>KeyError: <InterfaceClass Products.PlonePAS.interfaces.plugins.ILocalRolesPlugin>
>```
>
>The problem is completely transient. PluginRegistry raises a KeyError if it can't find any plugins for the given type, but I quite clearly have the Plone default borg_localroles. Indeed on a refresh the page loads without issue. What could make it unable to find the plugin temporarily??

I would install `Products.PDBDebugMode`. If your instance runs in
development mode, then an exception as the above would enter the
Python debugger and you can analyse the error producing state in
detail.

>The traceback here shows it is trying to get the local roles of some context to update that index in the catalog. This happens by calling processQueue() first which finishes any pending indexing as part of catalog optimization since 5.1. Is this supposed to (potentially) involve indexing from other requests?

Not unless you do very strange things with your requests/transactions.
Usually, `processQueue` is called in a preliminary step
of a transaction `commit`. As there is usually a tight coupling
between request and transaction, the queue is empty at the start
of a new request.

You might want to verify that the queue is
cleared when the transaction is aborted. Otherwise, a transaction abort
might introduce strange effects in a following transaction.

Plone Foundation Code of Conduct