Does collective.dexteritytextindexer.converters miss a convert for NamedBlobFile field

1

I have an add-on working on plone 5.2.4 with py27 that relies on collective.dexteritytextindexer 2.2.1 for a searchable field thesis:

    ...
    dexteritytextindexer.searchable("thesis")
    thesis = namedfile.NamedBlobFile(
        title=_(u'label_thesis', default=u'Thesis'),
        description=_(u'help_thesis',
                      default=u"Add your thesis file."),
        required=False,
    )
    ...

I migrated this addon to py37. Collective.dexteritytextindexer 2.2.4 has been installed. When I tried to add a new object with a pdf thesis, I got the following error message:

Collective module.dexteritytextindexer.indexer, line 94, in dynamic_searchable_text_indexer

AssertionError: Expected converted value of IDExterityTextIndexFieldConverter to be a str

After debugging I found that when changing line 128 of collective.dexteritytextindexer/converters.py at master · collective/collective.dexteritytextindexer · GitHub from
return datastream.getData() to return str(datastream.getData()) it is possible to add a new dexterity object with the pdf thesis.

What are the procedures so that the maintainers of the colective.dexteritytextindexer can validate this conversion to str or perform a more adequate one and launch a new version of the add-on?

2

Good catch! Recently I found PDF indexing not working in Plone core too, so this might be unreleated to the addon.

datastream.getData should return the correct type (str) here. I moreover see the source of this issue in Products.PortalTransforms.

Looking further the transform itself is defined here Products.PortalTransforms/pdf_to_text.py at master · plone/Products.PortalTransforms · GitHub
convertTo is here Products.PortalTransforms/TransformEngine.py at master · plone/Products.PortalTransforms · GitHub
It should return an IDataStream object, which on getData should return a str:
Products.PortalTransforms/interfaces.py at master · plone/Products.PortalTransforms · GitHub

Do you think you can go deeper into this rabbit hole?

3

Wow, was looking at the tip of the iceberg
I don't think i have the knowledge to fix it but i can definitely try. I am using plone 5.2.4 with py37. I'll insert some breakpoints to try to understand.

4

On my installation, the pdf_to_text transform doesn't fail. if the text type not really an str then an error occurs, investigate here: Products.PortalTransforms/commandtransform.py at master · plone/Products.PortalTransforms · GitHub