Error when searching for documents

Plone Support
1

Hi again!

Any idea what this is about, this worked yesterday 2020-03-31, and will work tomorrow, 2020-04-02 (tested that, but today I get this error, any idea why?

2020-04-01 13:20:31,176 ERROR   [Zope.SiteErrorLog:251][waitress] 1585740031.17279960.2274426737713554 http://localhost:8080/Plone/@@process_queued_orders
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 156, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 338, in publish_module
  Module ZPublisher.WSGIPublisher, line 256, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 62, in call_object
  Module plone.addon.browser.process_queued_orders, line 30, in __call__
  Module plone.addon.browser.process_queued_orders, line 59, in __has_ordered_today
  Module Products.CMFPlone.CatalogTool, line 457, in searchResults
  Module Products.ZCatalog.ZCatalog, line 611, in searchResults
  Module Products.ZCatalog.Catalog, line 1091, in searchResults
  Module Products.ZCatalog.Catalog, line 634, in search
  Module Products.ZCatalog.Catalog, line 564, in _search_index
  Module Products.ZCTextIndex.ZCTextIndex, line 210, in query_index
  Module Products.ZCTextIndex.ParseTree, line 132, in executeQuery
  Module Products.ZCTextIndex.BaseIndex, line 217, in search_phrase
TypeError: a bytes-like object is required, not 'str'

Our code that invokes the error

    57	brains = catalog.searchResults(**{
    58		'portal_type': 'Document',
    59		'Title': 'Orderdetails: ' + str(datetime.date.today()) #2020-04-01
    60	})
2

Also saw this in the logs

2020-04-01 14:16:03,057 ERROR   [Zope.SiteErrorLog:251][waitress] 1585743363.05721760.9802461614953288 http://localhost:8080/Plone/@@getVocabulary
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 156, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 338, in publish_module
  Module ZPublisher.WSGIPublisher, line 256, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 62, in call_object
  Module plone.app.content.browser.vocabulary, line 216, in __call__
  Module plone.app.vocabularies.catalog, line 571, in __iter__
  Module plone.memoize.instance, line 53, in memogetter
  Module plone.app.vocabularies.catalog, line 561, in brains
  Module Products.CMFPlone.CatalogTool, line 457, in searchResults
  Module Products.ZCatalog.ZCatalog, line 611, in searchResults
  Module Products.ZCatalog.Catalog, line 1091, in searchResults
  Module Products.ZCatalog.Catalog, line 634, in search
  Module Products.ZCatalog.Catalog, line 564, in _search_index
  Module Products.ZCTextIndex.ZCTextIndex, line 210, in query_index
  Module Products.ZCTextIndex.ParseTree, line 132, in executeQuery
  Module Products.ZCTextIndex.BaseIndex, line 217, in search_phrase
TypeError: a bytes-like object is required, not 'str'
3

Take the debugger and see what the data in line 217 is causing this error.

1 Like
4

This looks like a programming error in Products.ZCTextIndex (part of Products.ZCatalog). I remember to have seen corresponding messages in the early days when ZCTextIndex were being made Python 3 compatible. Alternatively, it is possible that something went wrong with the data migration from Python 2 to Python 3. Clear and rebuild the ZCTextIndexes to exclude this possibility.

Background: ZCTextIndex does not directly work with the words themselves but with an integer representation (called wid for "WordID"). When you search for a phrase, you are looking for documents that contain a sequence of words. ZCTextIndex maintains for each document the words it contains (--> _docwords[docid]). Logically, this is a sequence of integers but for efficiency reasons (less space in ZODB) it is represented as a (single) string. For this each integer is interpreted as unicode code point and is thereby associated with a (unicode) character. This way a sequence (of sufficiently small) integers can be transformed into an equivalent string. The word comprising the phrase to search for are encoded in the same way. The error you observe results from a discrepancy between the two encodings (one is using str, the other bytes) -- either a programming error or a data error.

5

OK, first thank you for the answers. I have some follow up questions.

Should I know how to " Clear and rebuild the ZCTextIndexes to exclude this possibility." ? Is there a guide or documentation somewhere?

And also "Take the debugger and see what the data in line 217 is causing this error." What debugger are you refeering to?

6

(portal-) ZMI --> portal_catalog --> Indexes. You will see there a list of indexes with their types. Select the ZCTextIndexes and press reindex (or similarly spelled) (this clears and rebuilds the selected indexes).

For analyzing problems resulting in exceptions, I like Products.PDBDebugMode (or similarly spelled). If your Plone runs in debug mode, this extensions enters the Python debugger (pdb --> Python library reference) for unhandled exceptions and error logs. In the debugger, you can then analyze the state around the problem.

7

The Python debugger "pdb".

import pdb; pdb.set_trace()
8

So, it looks like the clear and reindex worked, the error message is now gone. Thank you for your input.