Cannot get AutocompleteFieldWidgeet to work with a source binder

FWIW we switched to using plone.app.z3cform's AjaxSelectFieldWidget awhile back (I want to say 5.0, which is when we switched fully to Dexterity). I don't think plone.formwidget.autocomplete ever worked correctly for us in Dexterity, but I didn't look into it much because AjaxSelectFieldWidget seemed like a better solution anyway. There's also a non AJAX version SelectFieldWidget which has a slightly different UI (and is not AJAX obviously). We use that on some small vocabularies, but for large vocabularies you want AjaxSelectFieldWidget.

These widgets are also used in Dexterity content types out of the box currently, while I believe plone.formwidget.automcomplete is no longer part of Plone core.

@jensens I just tried but with no luck. Same error. :cry:

@Esoth I tried do switch to AjaxSelectFieldWidget:

from plone.app.z3cform import AjaxSelectFieldWidget

directives.widget(
        'city',
        AjaxSelectFieldWidget,
        source=ItalianCitiesSourceBinder()
    )
    city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source=ItalianCitiesSourceBinder(),
        required=False,
    )

but i got a RecursionError:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 255, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 61, in call_object
  Module plone.z3cform.layout, line 63, in __call__
  Module plone.z3cform.layout, line 47, in update
  Module plone.dexterity.browser.add, line 141, in update
  Module plone.z3cform.fieldsets.extensible, line 64, in update
  Module plone.autoform.form, line 34, in updateFields
  Module plone.autoform.base, line 67, in updateFieldsFromSchemata
  Module plone.dexterity.browser.base, line 25, in schema
  Module plone.dexterity.fti, line 269, in lookupSchema
  Module plone.alterego.dynamic, line 29, in __getattr__
  Module plone.synchronize.decorator, line 9, in synchronized_function
  ...
  Module plone.dexterity.schema, line 366, in __call__
  Module plone.dexterity.fti, line 281, in lookupModel
  Module plone.dexterity.fti, line 269, in lookupSchema
  Module plone.alterego.dynamic, line 29, in __getattr__
  Module plone.synchronize.decorator, line 9, in synchronized_function
RecursionError: maximum recursion depth exceeded while calling a Python object

I'm not sure if AjaxSelectFieldWidget supports the source parameter for a source binder or if it expects only a vocabulary.

I made an error on the import of AjaxSelectFieldWidget :man_facepalming:.
The correct import is:

from plone.app.z3cform.widget import AjaxSelectFieldWidget

However I keep getting the same error as with AutocompleteFieldWidget:

Traceback (innermost last):
...
ValueError: value or token must be provided (only one of those)

Might be a silly question, but PrincipalsVocabulary seems to be involved when that's not implied by your question. Is that deliberate or accidental?

@djowett thank you for answering.

Tere's no reference to PrincipalsVocabulary in the code I wrote. I cannot explain why it's in the stack trace. Maybe it's called from dexterity or z3cform for permission checks? I have no idea...

If I use a named vocabulary utility, the widget works fine.

Try ```
source=ItalianCitiesSourceBinder,

As suggested by @Esoth I replaced AutocompleteFieldWidget with AjaxSelectFieldWidget, so the updated code is the following:

If I remove those parenthesis:

city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source=ItalianCitiesSourceBinder,
    )

i get an InvalidVocabulary error:

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 155, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 337, in publish_module
  Module ZPublisher.WSGIPublisher, line 243, in publish
  Module ZPublisher.BaseRequest, line 523, in traverse
  Module ZPublisher.BaseRequest, line 350, in traverseName
  Module Products.CMFPlone.browser.admin, line 48, in publishTraverse
  Module ZPublisher.BaseRequest, line 144, in publishTraverse
  Module ZODB.Connection, line 795, in setstate
  Module ZODB.serialize, line 633, in setGhostState
  Module ZODB.serialize, line 626, in getState
  Module ZODB.serialize, line 483, in find_global
  Module ZODB.DB, line 864, in classFactory
  Module ZODB.broken, line 204, in find_global
  Module italian.cities.content.city, line 11, in <module>
  Module italian.cities.content.city, line 17, in ICity
  Module zope.schema._field, line 467, in __init__
zope.schema._field.InvalidVocabularyError: Invalid vocabulary <class 'italian.cities.sources.ItalianCitiesSourceBinder'>

If you go with this named vocabulary I think your zcml needs to be

<utility
factory=".vocabularies.CitiesVocabularyFactory"
name="italian.cities.list"
/>

(change CitiesVocabulary to CitiesVocabularyFactory) and then below your cities vocabulary class you can define it as

CitiesVocabularyFactory = CitiesVocabulary()
A factory is a design pattern for creating objects, in this case vocabularies.

I would not expect this to work. If you end up using a class that implements IContextSourceBinder instead of doing the named vocabulary like you have above, the source attribute of the field needs to be an instance of that class, not the class.

@Esoth the code i posted above (the one with the vocabulary) works fine, but it was only to test if my code worked at least with normal vocabularies. I created a branch vocab with that code:

The fact is I can't use a vocabulary because the data for the AjaxSelectFieldWidget comes from an external database and the number of records is very high. That's why I need to use a source binder. To my knowledge there's no way to make this work with vocabularies unless I fetch all the records from the database...

This error message indicates that your vocabulary does not provide the required interface. Look at the source code (--> zope.schema._field near line 467) to find out which interface is required and look for ways how to fulfill the requirement.

Hi @dieter thank you for your reply.

That error gets raised if I remove parenthesis on ItalianCitiesSourceBinder call in the source parameter as @djowett suggested. If I leave the parenthesis the error I get is the same of the initial post.
So what I'm trying to solve is the ValueError: value or token must be provided (only one of those).

Same principle: look at the traceback, locate the source raising the exception (and in your case likely its caller). The error message above suggests a wrong (vocabulary) Term construction: apparently, it wants either a token or a value but not both. Verify that the parameters are passed correctly.

That's what I did in the last few days, but the problem is that in the stack trace there's no code involved from my add-on. That's why I have difficulties debugging the code.
I doubt the error comes from my vocabulary (which looks ok to me) and as I already said, the same exact vocabulary works fine if used with a vocabulary factory and not as a Query Source for a Source Binder:

vocabulary = SimpleVocabulary((
        SimpleTerm(u'Bologna', 'bologna', u'Bologna'),
        SimpleTerm(u'Roma', 'roma', u'Roma'),
        SimpleTerm(u'Milano', 'milano', u'Milano'),
        SimpleTerm(u'Palermo', 'palermo', u'Palermo'),
        SimpleTerm(u'Sorrento', 'sorrento', u'Sorrento')))

@djowett there is some discussion of PrincipalsVocabulary here:

I am very surprised that you use the same vocabulary in both contexts: a SimpleVocabulary has by definition a fixed number of terms; a "Query Source" is quite different.

When I must analyze difficult situations involving exceptions, I use Products.PDBDebugMode. (If Plone runs in "debug mode",) it enters the Python debugger in the context of the exception and I can (quite easily) analyze what goes wrong and where the problematic values comes from.

I have checked out your repo and replace the SourceBinderVocabulary with:

<!-- configure.zcml -->
<utility
  component=".sources.italian_cities_factory"
  name="italian-cities"
  provides="zope.schema.interfaces.IVocabularyFactory"
  />
# source.py
from zope.interface import implementer
from zope.schema.vocabulary import SimpleTerm
from zope.schema.vocabulary import SimpleVocabulary


def italian_cities_factory(context):

    return SimpleVocabulary((
        SimpleTerm('Bologna', 'bologna', 'Bologna'),
        SimpleTerm('Roma', 'roma', 'Roma'),
        SimpleTerm('Milano', 'milano', 'Milano'),
        SimpleTerm('Palermo', 'palermo', 'Palermo'),
        SimpleTerm('Sorrento', 'sorrento', 'Sorrento')))
# city.py
from plone.autoform import directives
from plone.dexterity.content import Item
from plone.supermodel import model
from plone.app.z3cform.widget import AjaxSelectFieldWidget
from zope.interface import implementer
from zope import schema


class ICity(model.Schema):
    """ Marker interface and Dexterity Python Schema for City
    """
    city = schema.Choice(
        title=u"City name",
        description=u"For example: Bologna, Roma, etc...",
        source="italian-cities",
    )
    directives.widget(
        'city',
        AjaxSelectFieldWidget,
        source="italian-cities"
    )

@implementer(ICity)
class City(Item):
    """
    """

and all is fine, no errors, no tracebacks.

@mtrebron thank you, I saw the discussion here. In the final post you said you worked around the issue by monkey patching toWidgetValue in plone.app.z3cform.converters.AjaxSelectWidgetConverter.
As I said in the initial post, the only way I found to get this working is by monkey patching the same method by adding the following code at the beginning:

if value == (None,):
    value = None

How did you patch it?

I have the same code working perfectly in production on a Plone 4.2 installation for five years now. Here you can read another example, and here another one.
There's also an example on the official documentation of a Source Binder with SimpleVocabulary.

Me too.

@1letter this is basically the same code I wrote a few posts before, but as I already said I cannot use vocabularies because I need to fetch data from an external database and the number of records is very high.

Davide

        if (not value) or (None in value):
            return self.field.missing_value

My issue vanished after changing my schema to the structure suggested in #18

I did forget that AjaxSelectFieldWidget does not do batching. So once you do a search the result is probably ok, but really you want batching/paging especially if your query string is empty. I actually brought this up awhile back Batching AjaxSelectFieldWidget - #2 by MrTango so batching is supported in select2 but not with the mockup implementation. It really is too bad this doesn't support batching because even if you get the Autocomplete widget to work you are lacking design consistency on forms that mix this and the Ajax select2 implementation.

Side note: https://dist.plone.org/release/5.2.1/versions.cfg pins a version of plone.formwidget.autocomplete that is incompatible with Plone 5. I don't know what repo governs this, or where to report it.

Sorry to revive this old thread. But I am trying fix the issue Can't search and add Relations when adding new content · Issue #14 · Sinar/popolo.contenttypes · GitHub

I tried to monkey patch the 'toWidgetValue' but ended up getting 'from_object' Key error on trying to save the dexterity field (politikus.extractives/extractive_concession.py at 58ee456217c9005c233643b95b5f18303c861dbb · Sinar/politikus.extractives · GitHub).

    <monkey:patch
      description="Fix z3c.form value error exceptions"
      class="plone.app.z3cform.converters.AjaxSelectWidgetConverter"
      original="toWidgetValue"
      replacement=".converters.toWidgetValue"
      />




# -*- coding: utf-8 -*-
from z3c.form import converter
import six

def toWidgetValue(self, value):
    if not value:
        return self.field.missing_value
    vocabulary = self.widget.get_vocabulary()
    tokenized_value = []
    for term_value in value:
        if vocabulary is not None:
            try:
                term = vocabulary.getTerm(term_value)
                tokenized_value.append(term.token)
                continue
            except (LookupError, ValueError):
                pass
        tokenized_value.append(six.text_type(term_value))
    return getattr(self.widget, 'separator', ';').join(tokenized_value)
KeyError

Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 168, in transaction_pubevents
  Module transaction._manager, line 257, in commit
  Module transaction._manager, line 134, in commit
  Module transaction._transaction, line 267, in commit
  Module transaction._transaction, line 333, in _callBeforeCommitHooks
  Module transaction._transaction, line 372, in _call_hooks
  Module Products.CMFCore.indexing, line 316, in before_commit
  Module Products.CMFCore.indexing, line 226, in process
  Module Products.CMFCore.indexing, line 48, in reindex
  Module Products.CMFCore.CatalogTool, line 368, in _reindexObject
  Module Products.CMFPlone.CatalogTool, line 356, in catalog_object
  Module Products.ZCatalog.ZCatalog, line 505, in catalog_object
  Module Products.ZCatalog.Catalog, line 369, in catalogObject
  Module Products.PluginIndexes.unindex, line 242, in index_object
  Module Products.PluginIndexes.unindex, line 287, in _index_object
  Module Products.PluginIndexes.unindex, line 225, in insertForwardIndexEntry
  Module z3c.relationfield.relation, line 89, in __lt__
  Module z3c.relationfield.relation, line 37, in from_path
  Module plone.app.relationfield.monkey, line 19, in get_from_object
KeyError: 'from_object'