Hello everyone, our testing matrix start failing recently for Plone 5.1 builds, can someone help us please?

More information here:

Running collective.cover.testing.collective.cover:Integration tests:
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.340 seconds.
  Set up in 9.700 seconds.
  Set up collective.cover.testing.Fixture in 4.652 seconds.
  Set up collective.cover.testing.collective.cover:Integration in 0.000 seconds.
    7/8 (87.5%)> /home/rodfersou/.projects/collective.cover.p5/src/collective/cover/tiles/
--> 103         searchable_text = six.text_type(data.getData(), 'utf-8')
    104         return searchable_text.strip()  # remove leading and trailing spaces

ipdb> data
< object at 0x7ff58c4dfb30>
ipdb> data.getData()
u' My document text... '
ipdb> six.text_type(data.getData(), 'utf-8')
*** TypeError: decoding Unicode is not supported

@hvelarde it is weird, the last change in this module was 7 months ago, all other changes was years ago.

it should be something in the last release 3.1.4 (2018-09-23)

The quick fix is to change collective.cover's code here:

searchable_text = data.getData()
if isinstance(searchable_text, six.binary_type):
    searchable_text = searchable_text.decode('utf-8')


from Products.CMFPlone.utils import safe_unicode
return safe_unicode(data.getData())

Unluckily the output of convertTo has changed in a non retrocompatible way in a minor release and nobody noticed this.

Apparently convertTo should return a str object in both Python 2 and 3.

That is very bad and I am sorry it happened but it is good that now we know.
Not really sure how to fix this easily and in a fast way upstream.


fortunately we have pretty good tests in collective.cover that had lead to discover early a couple of issues on the core in the past :slight_smile:.

IMO, Products.PortalTransforms should be fixed before releasing Plone 5.2a1.

1 Like

@rodfersou are you able to create a pull request with a failing test in ? ... since core tests were passing we might have missed something there.

I'd recommend following best practice for transforming RichTextValue here ...


But this method was not changed in the last release.

the convertTo method returns an object implementing IDataStream ... so in detail, the text/plain transformer is a subclass of Products.PortalTransforms.libtransforms.retransform.retransform and this returns always a six.text_type since the last change of @pbauer ( ... maybe the reason why your transform doesn't work anymore.

@pigeonflight I think this is the change you're fighting with in your thread.

So maybe converge here as the root cause is now known.

My quick fix was to pin
Products.PortalTransforms = 3.1.2

Well spotted @petschki!
In this case the fix might be somebody has time to open a PR he is welcome.
Otherwise I will do one later/tomorrow

1 Like

Here it is
I modified the tests to ensure that the output type is not going to change in the future.


JFTR, Products.PortalTransforms 3.1.5 has been released and it solves the issue:

thanks, @alert, you rock!


Plone Foundation Code of Conduct