Select2 pattern not showing options for anonymous users

Hi all,

On my Plone 5.2 instance, I developed a standalone form following https://docs.plone.org/develop/addons/schema-driven-forms/index.html The form includes a select field that uses a vocabulary.

While the user is logged in, the select field loads its options from the vocabulary using mockup magic (great!). But when logged out, the magic does not happen. No select2-container div with the choices is inserted above the input field, apparently causing the field to then display as a standard text input...

While logged in, there is an explicit request for http://localhost:8000/Plone/++resource++mockup/base/pattern.js I don't see it in the anonymous request although pattern.js can be loaded anonymously.
@@getVocabulary?name=my_theme.MyVocabulary&field=my_field is also accesible for anonymous users. Not sure if these two data points are relevant.

I fail to understand what is supposed to insert the select2-container element, and which next steps I could take to fix... Any hints are much appreciated!

Here are the relevant HTML snippets in each case (reformatted for readability). All other parts are identical.

Logged in:

<div class="fieldErrorBox"></div>
<div class="select2-container select2-container-multi 
            pat-select2 text-widget choice-field select2-orderable"
     id="s2id_autogen5">
  <ul class="select2-choices">
    <li class="select2-search-field">
      <label for="s2id_autogen6" class="select2-offscreen"></label>
      <input type="text" autocomplete="off" autocorrect="off" 
             autocapitalize="off" spellcheck="false" class="select2-input" 
             id="s2id_autogen6" style="width: 10px;" placeholder="">
    </li>
  </ul>
</div>
<input class="pat-select2 text-widget choice-field" type="text" 
       name="form.widgets.my_field" value="" 
       data-pat-select2="{&quot;separator&quot;: &quot;;&quot;, 
         &quot;vocabularyUrl&quot;: &quot;http://localhost:8000/Plone/@@getVocabulary?name=my_theme.MyVocabulary
         &amp;field=my_field&quot;, &quot;maximumSelectionSize&quot;: 1, 
         &quot;allowNewItems&quot;: &quot;false&quot;, &quot;orderable&quot;: true}" 
       tabindex="-1" style="display: none;">

Logged out:

<div class="fieldErrorBox"></div>
<input class="pat-select2 text-widget choice-field" 
       name="form.widgets.my_field" value="" 
       data-pat-select2="{&quot;separator&quot;: &quot;;&quot;, 
         &quot;vocabularyUrl&quot;: &quot;http://localhost:8000/Plone/@@getVocabulary?name=my_theme.MyVocabulary
         &amp;field=my_field&quot;, &quot;maximumSelectionSize&quot;: 1, 
         &quot;allowNewItems&quot;: &quot;false&quot;, &quot;orderable&quot;: true}" 
       type="text">

It seems mockup-patterns-select2 has been moved from anonymous to logged-in bundle in Plone 5.2: https://github.com/plone/plone.staticresources/commit/fd1b720521684ce633e63d5d876025a80c240fb5#diff-326503c04befd43af4f54ae76ed27bb8

Therefore the table in README needs an update: https://github.com/plone/plone.staticresources#entry-points

You can test if that is your problem by going to:

  1. Site Setup
  2. Resource Registries
  3. On the left column, click on plone bundle
  4. On the right column, add mockup-patterns-select2 to Resources field
  5. Save

Thanks for the hints! Unfortunately, adding mockup-patterns-select2 to the plone pattern bundle had no visible effect.

When I try to recompile the bundle, the modal window bundle builder displays an infinite spinner. The compilation does not complete. After clicking the compile button, there is a 404 error for:http://localhost:8000/Plone/++plone++static/components/r.js/dist/r.js

Pity compilation TTW (through the web) does not seem to work. If you are able to reproduce this bug on a fresh Plone site (without 3rd party addons), could you please file an issue? https://github.com/plone/Products.ResourceRegistries/issues/new

Alternatively you can compile your bundle in the command line. After you save the change of including select2 pattern in the plone bundle, stop your instance and run ./bin/plone-compile-resources -b plone, then start it up again and make sure you have "Development Mode" disabled in Resource Registries so Plone serves the bundles from your filesystem (and not the ones generated for compilations TTW)

This is getting more interesting :wink: by the minute....

Although I have /usr/local/Plone52/zeocluster/eggs/plone.staticresources-1.2.0-py3.7.egg there is no plone-compile-resources in my ./zeocluster/bin folder

Moreover, the issue I described above regarding TTW compilation is said to have been fixed here: https://github.com/plone/Products.CMFPlone/issues/2969

Maybe @thet can provide some additional wisdom?

I just upgraded my Plone install from the Volto training in Ferrara, which is based on the unifiedinstaller and does not contain any third party addons. I can still reproduce this bug in a new Plone site. Will file an issue...

1 Like

@mtrebron regarding the missing plone-compile-resources script - did you follow the instructions described here? https://github.com/plone/plone.staticresources#generating-the-plone-compile-resources-script

Did you also run all upgrade steps? That should fix the issues with TTW compilation and the missing r.js script also. Especially the "Update the last_compilation date of all bundles." upgrade step from plone.staticresources (profile version 1 to 2) should fix it (or re-importing the whole plone.staticresources default bundle).

in plone.staticresources 1.2.0 we moved the select2 pattern from the plone bundle to plone-logged-in because it wasn't used in core Plone for anonymous users. You need to re-add it to the plone bundle and recompile. It might make sense to create a seperate bundle for the select2 pattern, so that activating it is just a click in the resourceregistry controlpanel (/cc @agitator ).

1 Like

Cheers for the followups.

Just to make sure I understand where to file my bugreport, as suggested by @davilima6 above: Products.ResourceRegistries is nowhere to be found in either of my installs. Its changelog says that the product was slated for removal in Plone 5.1. I presume you meant plone.staticresources?

Regarding the missing plone-compile-resources script:

[zopepy]
recipe = zc.recipe.egg
eggs =
    ${buildout:eggs}
interpreter = zopepy
scripts =
    zopepy
    plone-compile-resources
  • to make sure the missing script was not due to a customization, I first thought of installing a fresh new Plone based on unifiedinstaller 5.2b1 - but then found my vanilla install from October which has the same... Here, buildout.cfg points to base.cfg for its defaults - I had to manually add plone-compile-resources to the zopepy section.

Not sure what I am doing wrong here: did not see any buildout errors, the script is not there...

With both installs, I went to each Plone site and ran the standard @@plone-upgrade step. In each install, I added a brand new Plone site. The TTW compilation error occurs in each of the 4 cases.

Here is the output of instance.log from running the upgrade steps again as suggested by @thet

INFO    [Products.GenericSetup.tool:1384][waitress] Importing profile profile-plone.staticresources.upgrades:2 with dependency strategy ignore.
INFO    [Products.GenericSetup.tool:1423][waitress] Applying main profile profile-plone.staticresources.upgrades:2
INFO    [GenericSetup.rolemap:98][waitress] Role / permission map imported.
INFO    [Products.CMFPlone.resources.browser.combine:121][waitress] Wrote combined JS bundle "default".
INFO    [Products.CMFPlone.resources.browser.combine:121][waitress] Wrote combined JS bundle "logged-in".
WARNING [Products.CMFPlone.resources.browser.combine:53][waitress] Could not find resource ++plone++static/plone-datatables-compiled.css. You may have to create it first.
INFO    [Products.CMFPlone.resources.browser.combine:156][waitress] Wrote combined CSS bundle "default".
INFO    [Products.CMFPlone.resources.browser.combine:156][waitress] Wrote combined CSS bundle "logged-in".
INFO    [Products.CMFPlone.resources.browser.combine:184][waitress] Finished bundle compilation.
INFO    [GenericSetup:1091][waitress] Ran upgrade step Update Bundles for profile plone.staticresources:default
INFO    [Products.GenericSetup.tool:1384][waitress] Importing profile profile-plone.staticresources.upgrades:3 with dependency strategy ignore.
INFO    [Products.GenericSetup.tool:1423][waitress] Applying main profile profile-plone.staticresources.upgrades:3
INFO    [GenericSetup.rolemap:98][waitress] Role / permission map imported.
INFO    [Products.CMFPlone.resources.browser.combine:121][waitress] Wrote combined JS bundle "default".
INFO    [Products.CMFPlone.resources.browser.combine:121][waitress] Wrote combined JS bundle "logged-in".
WARNING [Products.CMFPlone.resources.browser.combine:53][waitress] Could not find resource ++plone++static/plone-datatables-compiled.css. You may have to create it first.
INFO    [Products.CMFPlone.resources.browser.combine:156][waitress] Wrote combined CSS bundle "default".
INFO    [Products.CMFPlone.resources.browser.combine:156][waitress] Wrote combined CSS bundle "logged-in".
INFO    [Products.CMFPlone.resources.browser.combine:184][waitress] Finished bundle compilation.
INFO    [GenericSetup:1091][waitress] Ran upgrade step Update bundles (split bundles) for profile plone.staticresources:default

The last compilation date of the plone bundle stays at 2019-10-28T12:36:00

Finally today, I tried using the current development version of plone.staticresources from github. Re-ran the upgrade steps again. This also made no difference.

I did notice something odd in https://github.com/plone/plone.staticresources/blob/master/src/plone/staticresources/static/resourceregistry-compiled.js and its minified version. They contain this line at the end:

define("/home/_thet/data/dev/plone/buildout.coredev/src/plone.staticresources/src/plone/staticresources/static/resourceregistry.js", function(){});

Update: I was able to generate a plone-compile-resources script by reading the zc.recipe.egg documentation and adding the following in my buildout:

parts =
    instance
    repozo
    backup
    zopepy
    unifiedinstaller
    precompiler
    setpermissions
    plone-compile-resources

...

[plone-compile-resources]
recipe = zc.recipe.egg:scripts
eggs = plone.staticresources
#interpreter = zopepy
entry-points = plone-compile-resources=plone.staticresources._scripts.compile_resources:main

running the script resulted in a traceback

(zinstance) administrator@kallisti:/usr/local/Plone52/zinstance$ sudo -u plone_buildout ./bin/plone-compile-resources -b plone 
Setup npm env
Running command: npm install
Traceback (most recent call last):
  File "./bin/plone-compile-resources", line 102, in <module>
    sys.exit(plone.staticresources._scripts.compile_resources.main())
  File "/usr/local/Plone52/zinstance/src/plone.staticresources/src/plone/staticresources/_scripts/compile_resources.py", line 140, in main
    subprocess.check_call(cmd)
  File "/usr/lib/python2.7/subprocess.py", line 536, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I am now running yarn upgrade - will report on progress if any...

Looks like I'm running into a npm related error now. I'm reading this as npm being unable to install jquery-form.form 4.2.2 but not sure how to solve and googled to no avail...

The debug log mentions:

308 silly fetchPackageMetaData error for jquery-form@git+https://github.com/jquery-form/form.git#4.2.2 Command failed: git checkout 4.2.2
308 silly fetchPackageMetaData error: pathspec '4.2.2' did not match any file(s) known to git.

Traceback:

Setup npm env
Running command: npm install
npm WARN deprecated hawk@3.1.3: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated cryptiles@2.0.5: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated sntp@1.0.9: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated hoek@2.16.3: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated boom@2.10.1: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm ERR! code 1
npm ERR! Command failed: git checkout 4.2.2
npm ERR! error: pathspec '4.2.2' did not match any file(s) known to git.
npm ERR! 

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/administrator/.npm/_logs/2019-12-29T21_10_33_830Z-debug.log
Traceback (most recent call last):
  File "/usr/local/Plone52/zinstance/bin/plone-compile-resources", line 102, in <module>
    sys.exit(plone.staticresources._scripts.compile_resources.main())
  File "/usr/local/Plone52/zinstance/src/plone.staticresources/src/plone/staticresources/_scripts/compile_resources.py", line 140, in main
    subprocess.check_call(cmd)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['npm', 'install']' returned non-zero exit status 1

Hi, Norbert.

Plone mockup package requires us to use yarn instead of npm (see changelog for version 3.0.0).

The error you got for jquery-form comes from that. Unfortunately we are fetching all Node packages from Github instead of from Npm's Registry - that was required for when we used Bower and hasn't been updated since we moved to Yarn. We need to update this for all packages, like it's done here for Sinon: https://github.com/plone/mockup/pull/914/commits/5ade3358913d9b9a0cb5eff02df19fe0fdc071a2

So I checked Github for jquery-form: the release 4.2.2 indeed does not exist, it should be v4.2.2. Yarn can handle such minor variation while Npm can't - thus it complains: pathspec '4.2.2' did not match any file(s) known to git

2 Likes

Hi @davilima6 thank you for your help so far.

The npm error described above happens when I run /usr/local/Plone52/zinstance/bin/plone-compile-resources -b plone

I have now added mockup to my local development eggs in addition to plone.staticresources. Trying to follow the commit you linked to, but fear that I am quickly getting out of my depth here.

Sorry for another long post ahead, merely trying to figure out what I am doing wrong...

/usr/local/Plone52/zinstance/src/mockup/yarn.lock now contains:

"jquery-form@https://github.com/jquery-form/form.git#v4.2.2":
  version "4.2.2"
  resolved "https://github.com/jquery-form/form.git#4beae57bc92402e2d03ca66b5ae2942d0c94c695"
  dependencies:
    jquery ">=1.7.2"

/usr/local/Plone52/zinstance/src/mockup/mockup/package-lock.json includes this section:

    "jquery-form": {
      "version": "git+https://github.com/jquery-form/form.git#4beae57bc92402e2d03ca66b5ae2942d0c94c695",
      "from": "git+https://github.com/jquery-form/form.git#v4.2.2",
      "requires": {
        "jquery": ">=1.7.2"
      },

In /usr/local/Plone52/zinstance/src/plone.staticresources/package.json I have

{
  "name": "plone.staticresource",
  "version": "1.0.0",
  "description": "Static resources for Plone",
  "repository": "git@github.com:plone/plone.staticresources.git",
  "license": "BSD-3-Clause",
  "dependencies": {
    "check-node-version": "^4.0.2",
    "jquery": "https://github.com/jquery/jquery-dist.git#1.12.4",
    "jquery-form": "v4.2.2",
    "less": "https://github.com/less/less.git#v2.7.3",
    "mockup": "https://github.com/plone/mockup.git",
    "requirejs": "^2.3.6"
  }
}

but strangely, /usr/local/Plone52/zinstance/src/plone.staticresources/yarn.lock includes

"jquery-form@https://github.com/jquery-form/form.git#4.2.2":
  version "4.2.2"
  resolved "https://github.com/jquery-form/form.git#4beae57bc92402e2d03ca66b5ae2942d0c94c695"
  dependencies:
    jquery ">=1.7.2"

jquery-form@v4.2.2:
  version "4.2.2"
  resolved "https://registry.yarnpkg.com/jquery-form/-/jquery-form-4.2.2.tgz#9f96fb141ec9cbe0cdaf58b4d3f1dcbb009cdd52"
  integrity sha512-HJTef7DRBSg8ge/RNUw8rUTTtB3l8ozO0OhD16AzDl+eIXp4skgCqRTd9fYPsOzL+pN6+1B9wvbTLGjgikz8Tg==
  dependencies:
    jquery ">=1.7.2"

In /usr/local/Plone52/zinstance/src/plone.staticresources/src/plone/staticresources/static/components/mockup/package.json I also see "jquery-form": "https://github.com/jquery-form/form.git#v4.2.2",

When running /usr/local/Plone52/zinstance/bin/plone-compile-resources -b plone the error remains the same as above.

What am I missing here?

No problem with the long post, more details help and I should have asked them earlier. Are you using a package to develop your site (e.g. like a theme or site policy package)? The most correct place to persist your changes is doing the work there, including an upgrade step to recompile the plone bundle after applying via code the changes I suggested you in the UI of Resources Registry (adding select2 resource to plone bundle).

But for now let's continue with this other route. I found a couple things weird in your files. It's worth doing a git reset --hard to discard any changes you have in mockup or git stash to set them aside for now. Better yet, remove mockup from your development eggs, you don't need it there unless you plan to open a PR to Plone or use a fork for your project, which is not recommended.

It seems you modified your /usr/local/Plone52/zinstance/src/plone.staticresources/package.json because you should not have a have a line with "jquery-form": "v4.2.2", but instead a version more similar (or same) as https://github.com/plone/plone.staticresources/blob/master/package.json. The hard reset should discard those modifications.

Recapitulating the idea is to:

  1. reconfigure your site's plone bundle to include select2 resource <-- I think you did this already, (after we succeed here we can do similarly and disable select2 in plone-logged-in to avoid loading it twice)
  2. enable buildout generation of plone-compile-resources script, as @thet described <-- also this seems to be done (isn't needed in a new installation but you are migrating/upgrading)
  3. run ./bin/plone-compile-resources -b plone to generate a new final bundle for JS and CSS files, resulting in 4 new files (CSS, JS and their sourcemaps) in src/plone.staticresources/src/plone/staticresources/static/plone-compiled*. I usually do git status once inside the src/plone.staticresources directory to spot such modified files.
  4. restart your instance(s) and Plone serves the version read from the filesystem (step 3 files), with disabled Development mode in Resources Registry

So no running Npm or Yarn manually in this case. Sorry to point in that direction in my last post: I got misguided by the Running command: npm install you reported earlier, which is normal/expected when running bin/plone-compile-resources. The pre-requisite of manually running Yarn instead of Npm is for when you are developing in src/mockup, which is not your need at the moment.

If you still don't get around it maybe we can meet at IRC or Plone Slack channel to get this done.

@davilima6 I was about to post an update and saw your new post.

Just earlier, I cloned the mockup repository and made the changes described above in a separate branch mtrebron-jquery-form I updated my buildout to use this new branch and updated package.json in plone.staticresources.

{
  "name": "plone.staticresource",
  "version": "1.0.0",
  "description": "Static resources for Plone",
  "repository": "git@github.com:plone/plone.staticresources.git",
  "license": "BSD-3-Clause",
  "dependencies": {
    "check-node-version": "^4.0.2",
    "jquery": "https://github.com/jquery/jquery-dist.git#1.12.4",
    "jquery-form": "v4.2.2",
    "less": "https://github.com/less/less.git#v2.7.3",
    "mockup": "https://github.com/mtrebron/mockup.git#mtrebron-jquery-form",
    "requirejs": "^2.3.6"
  }
}

This resulted in :smiley:

administrator@kallisti:/usr/local/Plone52/zinstance/src/plone.staticresources$ yarn upgrade
yarn upgrade v1.21.1
error Outdated lockfile. Please run `yarn install` and try again.
info Visit https://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.
administrator@kallisti:/usr/local/Plone52/zinstance/src/plone.staticresources$ yarn install
yarn install v1.21.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 46.93s.

Then I was able to run
administrator@kallisti:/usr/local/Plone52/zinstance$ ./bin/plone-compile-resources -b plone

without errors. I have yet to start up Plone, but from the looks of it I now should have a newly built bundle.

Considering that the Plone installation I have been doing this on was the barebones one I created in Ferrara, I think I should look at incorporating my newly found knowledge in the installation which won't display the select widget to anonymous visitors...

I still do not understand parts of the original issue:

  1. not being able to run a TTW bundle compilation despite using the latest version of plone.staticresources and having gone through all the upgrade steps
  2. not having the plone-compile-resources script installed by default even though the commands are in my zopepy section
  3. the issue with jquery-form v4.2.2 not being recognized when I first ran plone-compile-resources

The three points above make me think that I should be filing these three points as an issue in plone.staticresources in the next few days. Also happy to assist with resolving it if possible.

Now to answer your most recent reply: yes, I have a standalone form in my package, which should let anonymous visitors select from a number of options which are pulled in with a dynamic vocabulary created via a plone.restapi call to an endpoint on another Plone server.

I was unaware that mockup-patterns-select2 is not part of the plone bundle (anymore). Which lead me to my original post. - I will certainly follow your suggestion to include this recompilation as an upgrade step in my package, as soon as I get any sort of recompiling to work (either ttw or via plone-compile-resources) on that Plone instance.

Yes, this is correct.

As I have not made any manual changes on the Plone instance with the original issue, I will follow your latest steps there and report back.

Great news! I wait for the final confirmation :wink:

  1. The TTW compilation is indeed broken, it has to do with what @thet mentioned about r.js. I filed a bug in https://github.com/plone/plone.staticresources/issues/58

  2. In buildout.coredev we don't need the extra [plone-compile-resources] you mentioned but it's worth opening an issue or maybe someone can clarify here why that extra step would be needed in your installation. Would it be possible to publish the code or a stripped-down minimal version?

  3. So you did not modify anything and in the second run it worked? If you can reproduce, yes, please file an issue but check if you don't have any changes in src/mockup/package.json (or remove it as dev egg)

No, sorry to confuse: I have two separate Plone installs. The original one where the select options would not display is quite customized. There, I have not modified anything in plone.staticresources. The second one, where I added plone.staticresources and my mockup fork, and which now runs with the newly compiled bundle is a vanilla Plone. Tomorrow, I'll try to implement what I have learned this week on my original one.

1 Like

Update,

I added the plone-compile-resources part to buildout.cfg on the Plone install with the original select issue and ran it without problems. However, the script does not pick up my addition of mockup-patterns-select2 from the control panel. I had to manually add this to ./eggs/plone.staticresources-1.2.0-py3.7.egg/plone/staticresources/static/plone.js

With that, the behavior of my select fields did indeed change. There is still a difference between how they are displayed for anonymous and logged in visitors.

First screen capture below is from the unthemed site as displayed initially (2nd and 3rd field) and after focusing on the field (1st field). Second capture is how the form displays to logged in viewers.

anon-form

logged-in-form

Does your Plone Site have an id of Plone? Otherwise you have to manually specify the site id with:
./bin/plone-compile-resources -b plone --site-id MyPloneSite. That's possibly why your UI changes weren't picked up. More info in ./bin/plone-compile-resources --help

Any errors on console for anonymous?

No, it has a different id. Which I specified with the s parameter. Without specifying it, plone-compile-resources returns a traceback.

Somehow, there is a request for

http://localhost:8080/Plone/@@my-form/@@getVocabulary?my_theme.MyVocabulary&field=application_areas

which results in lookup not allowed

rather than the expected

http://localhost:8080/Plone/@@getVocabulary?my_theme.MyVocabulary&field=application_areas

which returns the expected json

The TTW building was indeed broken. See: https://github.com/plone/plone.staticresources/issues/58

2 Likes