Disable HTML filtering setting broken on 5.2.0

Plone 5.2.0. I want to allow editors to paste <script> tags into content.

  1. I follow the documentation to disable HTML filtering and safe HTML transformation.
  2. Step 1 is incorrect and is now managed through the Control Panel > Security > HTML Filtering.
  3. In HTML Filtering, I tick the box for "Disable HTML filtering", and Save.

Now when I try to edit and save any page, with or without a <script> tag, the following error is displayed.

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 255, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 61, in call_object
  Module zope.browserpage.simpleviewclass, line 41, in __call__
  Module Products.Five.browser.pagetemplatefile, line 126, in __call__
  Module Products.Five.browser.pagetemplatefile, line 61, in __call__
  Module zope.pagetemplate.pagetemplate, line 135, in pt_render
  Module Products.PageTemplates.engine, line 88, in __call__
  Module z3c.pt.pagetemplate, line 173, in render
  Module chameleon.zpt.template, line 306, in render
  Module chameleon.template, line 209, in render
  Module chameleon.template, line 187, in render
  Module 97631ed0669c933e3ee09d01c204d932.py, line 253, in render
  Module f134ef2d079ee9bf713e4cca9cddd628.py, line 1028, in render_master
  Module f134ef2d079ee9bf713e4cca9cddd628.py, line 345, in render_content
  Module 97631ed0669c933e3ee09d01c204d932.py, line 243, in __fill_content_core
  Module 97631ed0669c933e3ee09d01c204d932.py, line 136, in render_content_core
  Module plone.app.textfield.value, line 114, in output_relative_to
  Module plone.app.textfield.transform, line 87, in __call__
TransformError: Error during transformation

 - Expression: "python:context.text.output_relative_to(view.context)"
 - Filename:   ... egg/plone/app/contenttypes/browser/templates/document.pt
 - Location:   (line 15: col 29)
 - Source:     ... ucture python:context.text.output_relative_to(view.context)"
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Expression: "provider:plone.abovecontentbody"
 - Filename:   ... egg/Products/CMFPlone/browser/templates/main_template.pt
 - Location:   (line 94: col 70)
 - Source:     ... 
                                     ^
 - Expression: "context/main_template/macros/master"
 - Filename:   ... egg/plone/app/contenttypes/browser/templates/document.pt
 - Location:   (line 6: col 21)
 - Source:     ... etal:use-macro="context/main_template/macros/master"
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Arguments:  repeat: {...} (0)
               template: <ViewPageTemplateFile - at 0x7ffa7c414850>
               views: <ViewMapper - at 0x7ffa832a7f50>
               modules: <_SecureModuleImporter - at 0x7ffa83540550>
               args: <tuple - at 0x7ffa8c13a050>
               here: <ImplicitAcquisitionWrapper front-page at 0x7ffa745a96e0>
               wrapped_repeat: <SafeMapping - at 0x7ffa6d47a680>
               user: <ImplicitAcquisitionWrapper - at 0x7ffa745a9c30>
               nothing: <NoneType - at 0x563c203ca560>
               container: <ImplicitAcquisitionWrapper front-page at 0x7ffa745a96e0>
               root: <ImplicitAcquisitionWrapper  at 0x7ffa74643050>
               request: <WSGIRequest - at 0x7ffa83328a90>
               traverse_subpath: <list - at 0x7ffa7cfaedd0>
               default: <object - at 0x7ffa8c057770>
               context: <ImplicitAcquisitionWrapper front-page at 0x7ffa745a96e0>
               view: <SimpleViewClass from /usr/local/plone-5.2/buildout-cache/eggs/plone.app.contenttypes-2.1.2-py2.7.egg/plone/app/contenttypes/browser/templates/document.pt document_view at 0x7ffa832a74d0>
               translate: <function translate at 0x7ffa83163500>
               macroname: master
               options: {...} (0)
               loop: {...} (0)
               target_language: <NoneType - at 0x563c203ca560>

Where do I start to fix this issue?

I'll also submit a PR for the docs issue.

I also looked at this documentation, but it seems far out of date.

https://docs.plone.org/adapt-and-extend/config/html-filtering.html#enabling-html-embed-codes

  1. There is no "checkbox next to “Insert/edit Media”".
  2. There is no "Stripped Tags" list.
  3. There is no "Custom Tags" list.

There should be a .cause on the TransformError, and there should be an error message in the error log with what is going wrong. See https://github.com/plone/plone.app.textfield/blob/9f1bee152652ab46a84850236ee523516d8a77df/plone/app/textfield/transform.py#L86-L87
If not, I would add a pdb in transform.py and see what is happening there.

My guess is that you're pasting in something with an encoding that's not the same as what is specified in value.encoding.

That was it. The editor pasted in a literal non-breaking space, \x{A0}. Thank you!

To verify, I also tested separately with the string räksmörgås and an emoji :beer:, and each test generated the same error.

This instance runs on Python 2.7, and the docs regarding Unicode in Plone only address Python 2. Does that mean this issue "goes away" in Plone on Python 3?

Actually the docs are wrong. I found that I do not need to disable HTML filtering. Here's what I did:

  1. Under Site Setup > HTML Filtering, move script from Nasty tags to Valid tags, and save.

No steps from https://docs.plone.org/develop/plone/forms/wysiwyg.html#disabling-html-filtering-and-safe-html-transformation are necessary.

This is closer to being correct, but still not quite right:

https://docs.plone.org/adapt-and-extend/config/html-filtering.html#enabling-html-embed-codes

Computer says yes: https://github.com/plone/plone.app.textfield/blob/9f1bee152652ab46a84850236ee523516d8a77df/plone/app/textfield/transform.py#L79-L80

I'm assuming that on python3 it already will be a (unicode) string, so we don't decode it.
You'll have to try and find out if that's correct :slight_smile: