Themefragment not working in Plone 6.0.0b2 (worked in Plone 5.2)

I have a themefragment that retrieves news items from a folder and displays them in a slideshow. It works fine on Plone 5.2 but gives an error in Plone 6.0.0b2.

Here it is:

<form>
<tal:block tal:define="
         mtool portal/portal_membership;
         can_edit python:mtool.checkPermission('Modify portal content', context);"
>
<div class="my-hero-block">

        <div class="my-slider-header">
          <div data-animation="slide" data-duration="500" data-infinite="1" class="slider w-slider">
            <div class="w-slider-mask">
           <metal:block tal:repeat="slide python:portal['slider'].restrictedTraverse('@@contentlisting')(portal_type='News Item')">
                
                <tal:block tal:define="has_link python:getattr(slide,'link',None)">
                <div class="my-slide-padding w-slide" style="transform: translateX(0px); opacity: 1;">
                <metal:status tal:condition="can_edit">
                                           <a href="${slide/getURL}/edit" 
                                         class="my-slider-state
                                                state-${slide/review_state}" style="color:black;z-index: 3000">${slide/review_state} [edit]</a>
                                       </metal:status>    
                     <metal:withoutlink tal:condition="not:has_link">
                <span class="my-slider-box w-inline-block">
                  <h1 class="heading-3">${slide/title}</h1>
                  <h4 class="my-slider-subheading">${slide/description}</h4>
                  </span>                
                    </metal:withoutlink>
                    <metal:withlink tal:condition="has_link">
                                    <a href="${slide/link}" class="my-slider-box w-inline-block">
                  <h1 class="heading-3">${slide/title}</h1>
                  <h4 class="my-slider-subheading">${slide/description}</h4>
                </a>
                                    
                                    </metal:withlink>
                                    
                </div>
                </tal:block>
            </metal:block>                   


              </div>
         
           
             <div class="w-slider-arrow-left">
              <div class="my-nav-arrow w-icon-slider-left"></div>
            </div>
            <div class="w-slider-arrow-right">
              <div class="my-nav-arrow w-icon-slider-right"></div>
            </div>
            
          </div>
        </div>

</div>
</tal:block>
</form>

Here's the error on Plone 6:

We’re sorry, but there seems to be an error…
Exception
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 167, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 376, in publish_module
  Module ZPublisher.WSGIPublisher, line 271, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 68, in call_object
  Module collective.themefragments.traversal, line 176, in __call__
Exception: zExceptions.NotFound: getURL

 - Expression: "slide/getURL"
 - Filename:   slider
 - Location:   (line 16: col 54)
 - Arguments:  template: <ZopePageTemplate at /Plone/slider>
               here: <PloneSite at /Plone>
               context: <PloneSite at /Plone>
               container: <PloneSite at /Plone>
               nothing: None
               options: {'args': ()}
               root: <Application at >
               request: <WSGIRequest, URL=https://8080-...138c7hn3z.ws-us65.gitpod.io/@@theme-fragment/slider>
               modules: <Products.PageTemplates.ZRPythonExpr._SecureModuleImporter object at 0x7f0b4aeee880>
               view: <collective.themefragments.traversal.FragmentView object at 0x7f0b41d4de20>
               portal_url: ...
               portal: <PloneSite at /Plone>
               user: <PropertiedUser 'admin'>
               default: <DEFAULT>
               repeat: <Products.PageTemplates.engine.RepeatDictWrapper object at 0x7f0b44a35d80>
               loop: {'slide': <Products.PageTemplates.engine.RepeatItem object at 0x7f0b41d77e80>}
               target_language: None
               translate: <function BaseTemplate.render.<locals>.translate at 0x7f0b41aaa160>
               attrs: {'href': '${slide/getURL}/edit', 'class': 'txbl-slider-state\n                                                state-${slide/review_state}', 'style': 'color:black;z-index: 3000'}
               mtool: <MembershipTool at /Plone/portal_membership>
               can_edit: 1
               slide: <plone.app.contentlisting.catalog.CatalogContentListingObject instance at /Plone/slider/test-news-item>
               has_link: None

My most educated guess is that this has something to do with changes in the way traversal works on Plone 6. Would appreciate some pointers on how to work around this.

@pigeonflight This might be related to Removed earlypatches/expressions.py. by mauritsvanrees · Pull Request #3567 · plone/Products.CMFPlone · GitHub

Does it work if you use href="${python:slide.getURL()}" instead? This should also be faster than path traversal.

Are Zope traversing checks documented somewhere?

When I use href="${python:slide.getURL()}", I get insufficient privileges and very little further explanation in the logs.

In an attempt to isolate the issue further, I tried ${python:slide}

The result was a different error:

TypeError
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 167, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 376, in publish_module
  Module ZPublisher.WSGIPublisher, line 271, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 68, in call_object
  Module collective.themefragments.traversal, line 173, in __call__
  Module Products.PageTemplates.ZopePageTemplate, line 279, in _exec
  Module Products.PageTemplates.ZopePageTemplate, line 343, in pt_render
  Module Products.PageTemplates.PageTemplate, line 81, in pt_render
  Module zope.pagetemplate.pagetemplate, line 133, in pt_render
  Module Products.PageTemplates.engine, line 365, in __call__
  Module z3c.pt.pagetemplate, line 176, in render
  Module chameleon.zpt.template, line 302, in render
  Module chameleon.template, line 215, in render
  Module chameleon.utils, line 53, in raise_with_traceback
  Module chameleon.template, line 192, in render
  Module 73f71486726736b393b5e1eefb530368, line 197, in render
  Module 73f71486726736b393b5e1eefb530368, line 82, in __quote
  Module chameleon.zpt.template, line 282, in translate
  Module z3c.pt.pagetemplate, line 163, in translate
  Module chameleon.i18n, line 67, in fast_translate
  Module plone.app.contentlisting.contentlisting, line 115, in __eq__
TypeError: ('Could not adapt', '<plone.app.contentlisting.catalog.CatalogContentListingObject instance at /Plone/committees/newsletter-committee/slider-messages/myslide-content>', <InterfaceClass plone.app.contentlisting.interfaces.IContentListingObject>)

 - Expression: "python:slide"
 - Filename:   slider
 - Location:   (line 13: col 23)
 - Arguments:  template: <ZopePageTemplate at /Plone/slider>
               here: <PloneSite at /Plone>
               context: <PloneSite at /Plone>
               container: <PloneSite at /Plone>
               nothing: None
               options: {'args': ()}
               root: <Application at >
               request: <WSGIRequest, URL=http://8080-...tb138c7hn3z.ws-us65.gitpod.io/Plone/@@theme-fragment/slider>
               modules: <Products.PageTemplates.ZRPythonExpr._SecureModuleImporter object at 0x7f1e9d0c5520>
               view: <collective.themefragments.traversal.FragmentView object at 0x7f1e90515b20>
               portal_url: 'http://8080-...138c7hn3z.ws-us65.gitpod.io/Plone'
               portal: <PloneSite at /Plone>
               user: <PropertiedUser 'admin'>
               default: <DEFAULT>
               repeat: <Products.PageTemplates.engine.RepeatDictWrapper object at 0x7f1e914d7880>
               loop: {'slide': <Products.PageTemplates.engine.RepeatItem object at 0x7f1e8a1c0d00>}
               target_language: None
               translate: <function BaseTemplate.render.<locals>.translate at 0x7f1e907c2ca0>
               attrs: {}
               mtool: <MembershipTool at /Plone/portal_membership>
               can_edit: 1
               slide: <plone.app.contentlisting.catalog.CatalogContentListingObject instance at /Plone/committees/newsletter-committee/slider-messages/myslide-content>

I think most of my fragments (I have 50–100) works with Plone 6 (I have not tested latest version, though).

I put almost all the logic in .py-files (don't think that matters, though)

In templates, I think you could use

item_url item/getURL|item/absolute_url

for most cases.

I notice that I had 'one (very) old fragment with similar syntax' (I think it uses the current folder if no folder is set).

 tal:define="folder_url request/form/folder_url|None;
 .....
 
 tal:repeat="item  context/?folder_url/@@contentlisting"

Thanks @espenmn... I'll try that out.

No errors this time, but also it didn't read the url from the object.
I simplified my snippet to look like this.

   <metal:block tal:repeat="slide python:portal['slider'].restrictedTraverse('@@contentlisting')(portal_type='News Item')">
                     <div tal:define="slide_url slide/getURL|slide/absolute_url;
                                      slide_title slide/title|slide/Title|None">
                       the url for ${slide_title} is ${slide_url}
                     </div>
               </metal:block>

Next step is to do this in a .py file. If that fails, I'll be shipping browser views :frowning:

On to working with .py files... I hope I can use the plone.api.

Is it possible to get some kind of interactive debugging... with themefragments?
I seem to recall being able to get an interactive repl with these templates. Maybe themefragments are more restricted and don't allow that like you can do with full blown chameleon templates :thinking:

Attempting to import plone.api in my themefragment .py files doesn't work. When I call the fragment I get "insufficient-privileges".

:walking_man:t6:goes to dust off browserviews

True: plone.api does not work, but usually there are workarounds.

Another thing that is not working is pdb.settrace(); which can be irritating when trying to debug. It might be possible to whitelist (in restrictedPython ??), try to search (here), I think it has been discussed

I think this is the package Plone 5.2.x / Python 3.x + collective.trustedimports
Not so sure it works with Python 3
collective.trustedimports

Out of curiosity: What do you need plone.api for ( I have missed it too, but I have managed without it)

@espenmn I'm trying to pull items from a collection and display them in a slider with the ability to click through and visit the links.
The fragment below worked flawlessly on Plone 5.2. It is because it is not working that I'm trying to find other ways to do it, including the use of plone.api.

Some of this is slightly off topic, but I mention it anyway:

  1. I make some fragments based on keyword (Subject), this is often an alternative to collections. If so, my fragment.xml would include

    <field name="keyword" type="zope.schema.Choice">
       <description/>
       <required>True</required>
       <title>Keyword</title>
       <vocabulary>plone.app.vocabularies.Keywords</vocabulary>
     </field>
    

And my fragment.py something like

         def get_items(self):
            data = self.data
            keyword = data['keyword']
            # if isinstance(u"", str):
            #    keyword.encode('ascii','ignore')
            sorton = 'modified'
            sort_order = 'descending'
            if 'sort_order' in data.keys():
                sort_order = str(data['sort_order'])
            if 'sort_on' in data.keys():
                sorton = data['sort_on']
            language = self.context.Language
            return self.context.portal_catalog(Subject=keyword, sort_on=sorton, sort_order=sort_order, Language=language)

and the fragment.pt something like

		<div tal:define="items view/get_items">

		<tal:check tal:condition="items">
			<div tal:repeat="item items"
  1. I often use 'linked folder' this way. fragment.xml (there should be some syntax to only include folders, too):

      <field name="linked_folder" type="zope.schema.Choice">
         <description>Choose Folder. Leave empty for current folder</description>
         <required>False</required>
         <title>Select a folder (or folderish object)</title>
          <source>collective.themefragments.tiles.CatalogSource</source>
      </field>
    

And the fragment.py something like:

def get_allitems(self):
   linked = self.data['linked_folder']

if linked:
    folder = self.context.portal_catalog(UID=linked)
    return folder[0].getObject()

selected = self.data['select_folder']

if selected:
    folder = self.context.portal_catalog(UID=selected)
    return folder[0].getObject()

return None

I think you can also define items like this if it is a collection:

     tal:define="folder view/get_allitems;
            items folder/@@contentlisting|context/@@contentlisting|None;

PS: I am not 'working on Plone' at the moment, so I have not checked (spelling etc) too much'.

UPDATE: I have made my own vocabulary in the example for folders or collection (since the other approach does not work with multilingual sites, so the xml also includes.

 <field name="select_folder" type="zope.schema.Choice">
  <description>Useful for Multilingual setups</description>
  <required>False</required>
  <title>Alternatively choose folder here</title>
  <vocabulary>medialog.dutchestheme.ShowFoldersVocabulary</vocabulary>
 </field>

For the vocabulary, I think it is something like:

    def ShowFoldersVocabulary(context):

        folders = api.content.find(portal_type=['Folder', 'Collection'] , sort_on='sortable_title')

        if folders:
            terms = [ SimpleTerm(value=folder.UID, token=folder.UID, title=format_title(folder)) for folder in folders ]
        return SimpleVocabulary(terms)

    directlyProvides(ShowFoldersVocabulary, IVocabularyFactory)

Thanks @espenmn
The fragment.xml and self.data is because you're using fragments with tiles/mosaic correct?

right
But you could also hard code them in the py or pt files, if you are using 'path' or 'keyword (Subject')' the syntax is 'identical'

In my case I may use my fragments "standalone" rather than as tiles, so I'll likely avoid the .xml data stuff.

If so, I thought that:

 <tal:repeat tal:repeat="item  context/foldername/@@contentlisting"> 

Would work, or you maybe just use:

def get_items(self):
    folder_path = '/something')
    
        return self.context.portal_catalog(portal_type='my_content_type', path={'query': folder_path,}, Language=language, sort_on='sort_on', sort_order='sort_order')

(if it is a folder)

Will give this a whirl shortly.