Cannot get AutocompleteFieldWidgeet to work with a source binder

Plone Support Add-on Development
, , , ,
1

Hi,
I am writing a Plone 5.2 (Python 3.7) add-on for a Dexterity content type with a Choice field. The field is supposed to have autocomplete because there is a huge amount of possible choices (2k+) fetched from an external database. I used the AutocompleteFieldWidgeet from plone.formwidget.autocomplete, but I cannot get it to work correctly.
The issue is during the creation of the content object (if I edit an already existing one, all works fine): when I start typing on the field I get the following error:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 243, in publish
  Module ZPublisher.BaseRequest, line 523, in traverse
  Module ZPublisher.BaseRequest, line 330, in traverseName
  Module zope.traversing.namespace, line 165, in namespaceLookup
  Module plone.z3cform.traversal, line 54, in traverse
  Module plone.dexterity.browser.add, line 141, in update
  Module plone.z3cform.fieldsets.extensible, line 65, in update
  Module plone.z3cform.patch, line 30, in GroupForm_update
  Module z3c.form.group, line 141, in update
  Module z3c.form.group, line 52, in update
  Module z3c.form.group, line 48, in updateWidgets
  Module z3c.form.field, line 277, in update
  Module plone.app.z3cform.widget, line 443, in update
  Module z3c.form.browser.text, line 36, in update
  Module z3c.form.browser.widget, line 171, in update
  Module Products.CMFPlone.patches.z3c_form, line 47, in _wrapped
  Module z3c.form.widget, line 132, in update
  Module plone.app.z3cform.converters, line 181, in toWidgetValue
  Module plone.app.vocabularies.principals, line 147, in getTerm
  Module plone.app.vocabularies.principals, line 113, in _get_term_from_source
ValueError: value or token must be provided (only one of those)

I wrote an example with a vocabulary similar to this. The code is in this github repository:

The relevant parts are the following:

The only way I found to fix this (as suggested here) is by monkey patching the method toWidgetValue of plone.app.z3cform.converters.AjaxSelectFieldWidget like this:

$ diff --color -U10 converters.py.bak converters.py
--- converters.py.bak	2020-04-27 15:01:56.209325019 +0200
+++ converters.py	2020-04-27 15:02:46.873695155 +0200
@@ -164,20 +164,22 @@
 
     def toWidgetValue(self, value):
         """Converts from field value to widget tokenized widget value.
 
         :param value: Field value.
         :type value: list |tuple | set
 
         :returns: Items separated using separator defined on widget
         :rtype: string
         """
+        if value == (None,):
+            value = None
         if not value:
             return self.field.missing_value
         vocabulary = self.widget.get_vocabulary()
         tokenized_value = []
         for term_value in value:
             if vocabulary is not None:
                 try:
                     term = vocabulary.getTerm(term_value)
                     tokenized_value.append(term.token)
                     continue

If possible I'd like to avoid monkey patching. Is there a better way to fix this or some other widget that I could use?

Thank you,
Davide

Previous posts that might be related:

2 Likes
2

Maybe - or not - this is related to the fix in plone.app.vocabularies 4.1.1: Did you try the latest bugfix release 4.1.2?

3

FWIW we switched to using plone.app.z3cform's AjaxSelectFieldWidget awhile back (I want to say 5.0, which is when we switched fully to Dexterity). I don't think plone.formwidget.autocomplete ever worked correctly for us in Dexterity, but I didn't look into it much because AjaxSelectFieldWidget seemed like a better solution anyway. There's also a non AJAX version SelectFieldWidget which has a slightly different UI (and is not AJAX obviously). We use that on some small vocabularies, but for large vocabularies you want AjaxSelectFieldWidget.

These widgets are also used in Dexterity content types out of the box currently, while I believe plone.formwidget.automcomplete is no longer part of Plone core.

4

@jensens I just tried but with no luck. Same error. :cry:

@Esoth I tried do switch to AjaxSelectFieldWidget:

from plone.app.z3cform import AjaxSelectFieldWidget

directives.widget(
        'city',
        AjaxSelectFieldWidget,
        source=ItalianCitiesSourceBinder()
    )
    city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source=ItalianCitiesSourceBinder(),
        required=False,
    )

but i got a RecursionError:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 255, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 61, in call_object
  Module plone.z3cform.layout, line 63, in __call__
  Module plone.z3cform.layout, line 47, in update
  Module plone.dexterity.browser.add, line 141, in update
  Module plone.z3cform.fieldsets.extensible, line 64, in update
  Module plone.autoform.form, line 34, in updateFields
  Module plone.autoform.base, line 67, in updateFieldsFromSchemata
  Module plone.dexterity.browser.base, line 25, in schema
  Module plone.dexterity.fti, line 269, in lookupSchema
  Module plone.alterego.dynamic, line 29, in __getattr__
  Module plone.synchronize.decorator, line 9, in synchronized_function
  ...
  Module plone.dexterity.schema, line 366, in __call__
  Module plone.dexterity.fti, line 281, in lookupModel
  Module plone.dexterity.fti, line 269, in lookupSchema
  Module plone.alterego.dynamic, line 29, in __getattr__
  Module plone.synchronize.decorator, line 9, in synchronized_function
RecursionError: maximum recursion depth exceeded while calling a Python object

I'm not sure if AjaxSelectFieldWidget supports the source parameter for a source binder or if it expects only a vocabulary.

5

I made an error on the import of AjaxSelectFieldWidget :man_facepalming:.
The correct import is:

from plone.app.z3cform.widget import AjaxSelectFieldWidget

However I keep getting the same error as with AutocompleteFieldWidget:

Traceback (innermost last):
...
ValueError: value or token must be provided (only one of those)
6

Might be a silly question, but PrincipalsVocabulary seems to be involved when that's not implied by your question. Is that deliberate or accidental?

7

@djowett thank you for answering.

Tere's no reference to PrincipalsVocabulary in the code I wrote. I cannot explain why it's in the stack trace. Maybe it's called from dexterity or z3cform for permission checks? I have no idea...

If I use a named vocabulary utility, the widget works fine.

8

Try ```
source=ItalianCitiesSourceBinder,

9

As suggested by @Esoth I replaced AutocompleteFieldWidget with AjaxSelectFieldWidget, so the updated code is the following:

If I remove those parenthesis:

city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source=ItalianCitiesSourceBinder,
    )

i get an InvalidVocabulary error:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 243, in publish
  Module ZPublisher.BaseRequest, line 523, in traverse
  Module ZPublisher.BaseRequest, line 350, in traverseName
  Module Products.CMFPlone.browser.admin, line 48, in publishTraverse
  Module ZPublisher.BaseRequest, line 144, in publishTraverse
  Module ZODB.Connection, line 795, in setstate
  Module ZODB.serialize, line 633, in setGhostState
  Module ZODB.serialize, line 626, in getState
  Module ZODB.serialize, line 483, in find_global
  Module ZODB.DB, line 864, in classFactory
  Module ZODB.broken, line 204, in find_global
  Module italian.cities.content.city, line 11, in <module>
  Module italian.cities.content.city, line 17, in ICity
  Module zope.schema._field, line 467, in __init__
zope.schema._field.InvalidVocabularyError: Invalid vocabulary <class 'italian.cities.sources.ItalianCitiesSourceBinder'>
10

If you go with this named vocabulary I think your zcml needs to be

<utility
factory=".vocabularies.CitiesVocabularyFactory"
name="italian.cities.list"
/>

(change CitiesVocabulary to CitiesVocabularyFactory) and then below your cities vocabulary class you can define it as

CitiesVocabularyFactory = CitiesVocabulary()
A factory is a design pattern for creating objects, in this case vocabularies.

I would not expect this to work. If you end up using a class that implements IContextSourceBinder instead of doing the named vocabulary like you have above, the source attribute of the field needs to be an instance of that class, not the class.

11

@Esoth the code i posted above (the one with the vocabulary) works fine, but it was only to test if my code worked at least with normal vocabularies. I created a branch vocab with that code:

The fact is I can't use a vocabulary because the data for the AjaxSelectFieldWidget comes from an external database and the number of records is very high. That's why I need to use a source binder. To my knowledge there's no way to make this work with vocabularies unless I fetch all the records from the database...

12

This error message indicates that your vocabulary does not provide the required interface. Look at the source code (--> zope.schema._field near line 467) to find out which interface is required and look for ways how to fulfill the requirement.

13

Hi @dieter thank you for your reply.

That error gets raised if I remove parenthesis on ItalianCitiesSourceBinder call in the source parameter as @djowett suggested. If I leave the parenthesis the error I get is the same of the initial post.
So what I'm trying to solve is the ValueError: value or token must be provided (only one of those).

14

Same principle: look at the traceback, locate the source raising the exception (and in your case likely its caller). The error message above suggests a wrong (vocabulary) Term construction: apparently, it wants either a token or a value but not both. Verify that the parameters are passed correctly.

15

That's what I did in the last few days, but the problem is that in the stack trace there's no code involved from my add-on. That's why I have difficulties debugging the code.
I doubt the error comes from my vocabulary (which looks ok to me) and as I already said, the same exact vocabulary works fine if used with a vocabulary factory and not as a Query Source for a Source Binder:

vocabulary = SimpleVocabulary((
        SimpleTerm(u'Bologna', 'bologna', u'Bologna'),
        SimpleTerm(u'Roma', 'roma', u'Roma'),
        SimpleTerm(u'Milano', 'milano', u'Milano'),
        SimpleTerm(u'Palermo', 'palermo', u'Palermo'),
        SimpleTerm(u'Sorrento', 'sorrento', u'Sorrento')))
16

@djowett there is some discussion of PrincipalsVocabulary here:

17

I am very surprised that you use the same vocabulary in both contexts: a SimpleVocabulary has by definition a fixed number of terms; a "Query Source" is quite different.

When I must analyze difficult situations involving exceptions, I use Products.PDBDebugMode. (If Plone runs in "debug mode",) it enters the Python debugger in the context of the exception and I can (quite easily) analyze what goes wrong and where the problematic values comes from.

18

I have checked out your repo and replace the SourceBinderVocabulary with:

<!-- configure.zcml -->
<utility
  component=".sources.italian_cities_factory"
  name="italian-cities"
  provides="zope.schema.interfaces.IVocabularyFactory"
  />

# source.py
from zope.interface import implementer
from zope.schema.vocabulary import SimpleTerm
from zope.schema.vocabulary import SimpleVocabulary


def italian_cities_factory(context):

    return SimpleVocabulary((
        SimpleTerm('Bologna', 'bologna', 'Bologna'),
        SimpleTerm('Roma', 'roma', 'Roma'),
        SimpleTerm('Milano', 'milano', 'Milano'),
        SimpleTerm('Palermo', 'palermo', 'Palermo'),
        SimpleTerm('Sorrento', 'sorrento', 'Sorrento')))

# city.py
from plone.autoform import directives
from plone.dexterity.content import Item
from plone.supermodel import model
from plone.app.z3cform.widget import AjaxSelectFieldWidget
from zope.interface import implementer
from zope import schema


class ICity(model.Schema):
    """ Marker interface and Dexterity Python Schema for City
    """
    city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source="italian-cities",
    )
    directives.widget(
        'city',
        AjaxSelectFieldWidget,
        source="italian-cities"
    )

@implementer(ICity)
class City(Item):
    """
    """

and all is fine, no errors, no tracebacks.

19

@mtrebron thank you, I saw the discussion here. In the final post you said you worked around the issue by monkey patching toWidgetValue in plone.app.z3cform.converters.AjaxSelectWidgetConverter.
As I said in the initial post, the only way I found to get this working is by monkey patching the same method by adding the following code at the beginning:

if value == (None,):
    value = None

How did you patch it?

I have the same code working perfectly in production on a Plone 4.2 installation for five years now. Here you can read another example, and here another one.
There's also an example on the official documentation of a Source Binder with SimpleVocabulary.

Me too.

@1letter this is basically the same code I wrote a few posts before, but as I already said I cannot use vocabularies because I need to fetch data from an external database and the number of records is very high.

Davide

20 
        if (not value) or (None in value):
            return self.field.missing_value

My issue vanished after changing my schema to the structure suggested in #18