Wrong Diazo transformation on Plone 5.2 Python 3.6

Plone Support Theming
1

When having a diazo theme that includes the following rule:
<replace css:theme="#theme-div" css:content="#css-div" href="@@browser-view"/>
The unicode chars from that view are not showed correctly, i.e: á shows like: Ã

It only happens when the html from the view its include with the rule, if you visit /@@browser-view
all characters are showed correctly, also if the characters come from the content.

Has anybody experienced this error? Any hint how to solve it?

2

If you use this for injecting CSS, I do the same with themefragments (and it works).

   <replace css:theme="#custcss" css:content="#custcss"  href="/@@theme-fragment/customcss"  />

(the /customcss means that the template has content
with id=customcss

You should also be able to 'include browser views' from themefragments… probably something like this:

<tal:replace  tal:replace="structure context/@@myview"></tal:replace>
3

If someone encounters the same issue this is what I have found after debuging diazo, plone.app.theming, and plone.transformchain. Seems the problem it is related with the encoding.

The transformation returns a repoze.xmliter.serializer.XMLSerializer object.
For their elements a decode('utf-8') it's applied.
Modifying it by decode('utf-8').encode('latin-1').decode('utf-8') fixed it for me.

I have filled an issue on the repo:

4

Seems I was wrong, the fix isn't working, when trying to decode the bytes are already mixed encoded content.
I have filled another issue on plone.app.theming: https://github.com/plone/plone.app.theming/issues/162

5

This is a wild guess, but I have had a lot of unicode errors with different versions of 'theming related stuff'.
For example: I can not use the default pins of plone.app.mosaic.

Could you try this and see if it changes anything:

6

Well I have taken a look and plone.jsonserializer does not come by default (clean installation). It is pulled by plone.app.mosaic

7

Sorry for the confusing answer.

I was thinking: Is there something similar in some of the code.

When I ported some of my add-ons to 5.2 I think I saw similar (unicode) errors several places (If I remember right they came from the way things were checked with six

if isinstance(url, six.text_type):
if isinstance(url, six.binary_type):

I had the same problems with plone.subrequest ( I need to pin it to 1.8.6 to get mosaic to work without unicode errors.

8

I got this issue as well, any solution or workaround in sight?

9

What I did was modify: https://github.com/plone/plone.app.theming/blob/master/src/plone/app/theming/utils.py#L167
Adding the following code:

elif content_type == 'text/html':
    result = ''.join([
         '<meta charset="utf-8"/>',
          result,
    ])

Also you can move the code from browsers views to viewlets so you don't need to inject it with diazo

10

Thank you @csanahuja your modification works fine. I will also look into if there is any chance I don't need to inject and replace content with diazo.