Attempt to update buildout to 5.1.2 breaks with "Error Could not import Products.PlonePAS"

Plone Support Administering Plone sites
,
1

Sorry if this is obvious somehow. I'm afraid my Plone chops are very rusty indeed at this point.

So, I'm simply rerunning buildout with the pointers updated to 5.1.2
I know it's not PlonePAS that's the problem. In fact, it was PloneFormGen until I commented that out.

Restarting shows:
/opt/plone/zeocluster$ sudo -u plone_daemon ./bin/plonectl fg client1
client1: 2018-05-12 20:02:50 INFO ZServer HTTP server started at Sat May 12 20:02:50 2018
Hostname: 0.0.0.0
Port: 8080
2018-05-12 20:02:53 ERROR Application Could not import Products.PlonePAS
Traceback (most recent call last):
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/OFS/Application.py", line 608, in import_product
product=import(pname, global_dict, global_dict, silly)
File "/opt/plone/buildout-cache/eggs/Products.PlonePAS-5.0.15-py2.7.egg/Products/PlonePAS/init.py", line 18, in
from Products.PlonePAS.tools.membership import MembershipTool
File "/opt/plone/buildout-cache/eggs/Products.PlonePAS-5.0.15-py2.7.egg/Products/PlonePAS/tools/membership.py", line 34, in
from plone.protect.interfaces import IDisableCSRFProtection
File "/opt/plone/buildout-cache/eggs/plone.protect-3.1.3-py2.7.egg/plone/protect/init.py", line 6, in
from plone.protect.utils import protect
SyntaxError: unqualified exec is not allowed in function 'call' it contains a nested function with free variables (utils.py, line 63)
Traceback (most recent call last):
File "/opt/plone/zeocluster/parts/client1/bin/interpreter", line 323, in
exec(compile(__file__f.read(), file, "exec"))
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/Startup/run.py", line 76, in
run()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/Startup/run.py", line 22, in run
starter.prepare()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/Startup/init.py", line 92, in prepare
self.startZope()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/Startup/init.py", line 268, in startZope
Zope2.startup()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/init.py", line 47, in startup
_startup()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/Zope2/App/startup.py", line 69, in startup
OFS.Application.import_products()
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/OFS/Application.py", line 585, in import_products
import_product(product_dir, product_name, raise_exc=debug_mode)
File "/opt/plone/buildout-cache/eggs/Zope2-2.13.27-py2.7.egg/OFS/Application.py", line 608, in import_product
product=import(pname, global_dict, global_dict, silly)
File "/opt/plone/buildout-cache/eggs/Products.PlonePAS-5.0.15-py2.7.egg/Products/PlonePAS/init.py", line 18, in
from Products.PlonePAS.tools.membership import MembershipTool
File "/opt/plone/buildout-cache/eggs/Products.PlonePAS-5.0.15-py2.7.egg/Products/PlonePAS/tools/membership.py", line 34, in
from plone.protect.interfaces import IDisableCSRFProtection
File "/opt/plone/buildout-cache/eggs/plone.protect-3.1.3-py2.7.egg/plone/protect/init.py", line 6, in
from plone.protect.utils import protect
File "/opt/plone/buildout-cache/eggs/plone.protect-3.1.3-py2.7.egg/plone/protect/utils.py", line 63
exec(_buildFacade(name, spec, callable.doc), facade_globs)
SyntaxError: unqualified exec is not allowed in function 'call' it contains a nested function with free variables

My Buildout currently:

############################################
#
# Buildout Configuration File for Plone
# -------------------------------------
#
# ALWAYS back up all Plone/Zope data and components
# before changing configuration.
#
# Running "bin/buildout" will update your installation,
# installing missing components as necessary.
#
# This will update the add-on products you've added in the eggs= lines.
# This will not, however, upgrade Plone itself (or anything else you've
# pinned with a version specification). To upgrade Plone itself, see the
# comments in "Plone Component Versions".
#
# Tutorial instructions for using zc.buildout for
# configuration management are available at:
# http://plone.org/documentation/tutorial/buildout
# Full details at http://pypi.python.org/pypi/zc.buildout
#
############################################

[buildout]
############################################
# Plone Component Versions
# ------------------------
# This version of the Unified Installer has the components of Plone 5
# preloaded so that it can install without an Internet connection.
# If you want to update, uncomment the "http://..." line below,
# edit it to point to the current version URL, comment out the
# "versions.cfg" line and run "bin/buildout" while attached to the
# Internet. Generally, you only want to do that as part of a planned migration.
# Note that if you are updating components, you should also check the versions
# section at the end of this file, since recipes or components other than
# those of Zope and Plone may need updating at the same time.
#
extends =
    base.cfg
#    versions.cfg
    http://dist.plone.org/release/5.1.2/versions.cfg

# If you change your Plone version, you'll also need to update
# the repository link below.
find-links +=
    http://dist.plone.org/release/5.1.2
    
index =https://pypi.python.org/simple/

# If you try to start Zope as root, it will change user id to run as
# the effective user specified here. This user id must own the var directory
# of your buildout.
effective-user = plone_daemon
# This user will own the rest of the installation, and should be used to
# run buildout.
buildout-user = plone_buildout
# A flag to tell the Unified Installer whether or not to document sudo use.
need-sudo = yes

############################################
# Eggs
# ----
# Add an indented line to the eggs section for any Python
# eggs or packages you wish to include in your Plone instance.
#
# Note that versions may be specified here or in the [versions]
# section below. You should always specify versions that you know
# are compatible with the Plone release and at an acceptable
# development level.
#
# If you update to a later version of Plone, remove the hotfix.
#
eggs =
    Plone
    Pillow
#    Products.PloneFormGen
    Products.RedirectionTool
    plone.app.mosaic
    plone.app.imagecropping

    collective.recaptcha
    plone.formwidget.recaptcha
    collective.plonetruegallery
    collective.ptg.unitegallery
    collective.ptg.carousel
    collective.ptg.highslide
    collective.ptg.contactsheet
    collective.ptg.presentation
    collective.ptg.fancybox
    collective.ptg.supersized    

    plonetheme.booster
    plonetheme.business_casual
   

############################################
# ZCML Slugs
# ----------
# Some eggs need ZCML slugs to tell Zope to
# use them. This is increasingly rare.
zcml =
    collective.recaptcha

############################################
# Development Eggs
# ----------------
# You can use paster to create "development eggs" to
# develop new products/themes. Put these in the src/
# directory.
# You will also need to add the egg names in the
# eggs section above, and may also need to add them
# to the zcml section.
#
# Provide the *paths* to the eggs you are developing here:
develop =
#    src/my.package

############################################
# var Directory
# -------------
# Sets the target directory for the "var" components of the install such as
# database and log files.
#
var-dir=${buildout:directory}/var

############################################
# Backup Directory
# ----------------
# Sets the target directory for the bin/backup and bin/snapshotbackup
# commands. Default is inside this project's var directory, but ideally
# this should be on a separate volume or backup server.
#
backups-dir=${buildout:var-dir}

############################################
# Initial User
# ------------
# This is the user id and password that will be used to create the initial
# user id that will allow you to log in and create a Plone site. This only
# sets the initial password; it will not allow you to change an already
# existing password. If you change the admin password via the web interface,
# the one below will no longer be valid.
# If you find yourself locked out of your Zope/Python installation, you may
# add an emergency user via "bin/plonectl adduser".
user=admin:

############################################
# Debug Options
# -------------
# Start Zope/Plone instances in "fg" mode to turn on debug mode;
# this will dramatically slow Plone.
#
# Add-on developers should turn deprecation warnings on
deprecation-warnings = off
# change verbose-security to "on" for useful security errors while developing
verbose-security = off

############################################
# Parts Specification
#--------------------
# Specifies the components that should be included in the buildout.
# Most are defined in the base.cfg extension; you may add your
# own if you need them at the end of this file.
parts =
    zeoserver
    client1
    backup
    zopepy
    unifiedinstaller
    precompiler
    setpermissions

############################################
# Major Parts
# ----------------------
# These common parts make use of sane base settings from
# base.cfg. To customize a part, just add whatever options
# you need. Read base.cfg for common settings.

[zeoserver]
<= zeoserver_base
recipe = plone.recipe.zeoserver
zeo-address = 127.0.0.1:8100

[client1]
<= client_base
recipe = plone.recipe.zope2instance
zeo-address = ${zeoserver:zeo-address}
http-address = 8080

############################################
# Versions Specification
# ----------------------
# Version information supplied here will "pin" Python packages to a particular
# version number, even when you use the "newest" flag running buildout.
# Specifying versions for all packages is a good idea and can prevent
# accidental changes when you add new packages to your buildout.
# Note that versions specified here will override those specified earlier
# in the configuration, including those from the Plone and Zope version
# config files.
#
[versions]
# Use the setuptools and zc.buildout versions that are
# available in our Python environment.
setuptools =
zc.buildout =

#tweak
# not for Plone 5.1 plone.app.tiles                       = 2.2.1
# not for Plone 5.1 plone.app.blocks =3.1.0

Products.PloneFormGen =1.8.5
Products.DocFinderTab = 1.0.5
bobtemplates.plone = 1.0.3
buildout.sanitycheck = 1.0.2
collective.checkdocs = 0.2
collective.recipe.backup = 3.0.0
mr.bob = 0.1.2
pkginfo = 1.3.2
plone.recipe.unifiedinstaller = 4.3.2
requests-toolbelt = 0.6.2
twine = 1.7.2
zest.pocompile = 1.4
zest.releaser = 6.6.4

# Required by:
# Jinja2==2.8
MarkupSafe = 0.23

# Required by:
# clint==0.5.1
args = 0.1.0

# Required by:
# twine==1.7.2
clint = 0.5.1

# Required by:
# zest.releaser==6.6.4
colorama = 0.3.7

### from polyester Paul Roeland
### For the record, that’s my pins on 5.1rc2, which work on Firefox (don’t have Safari, Linux user…)
plone.tiles = 2.0.0b3
plone.subrequest = 1.8.1
plone.app.tiles = 3.0.3
plone.app.standardtiles = 2.2.0
plone.app.blocks = 4.1.1
plone.app.drafts = 1.1.2
plone.app.mosaic = 2.0rc8
plone.formwidget.multifile = 2.0
plone.jsonserializer = 0.9.5
2

yes this is baffling; it's a compile error (no code is executed at this level). It should not depend on a buildout or addons, it should appear for everyone trying to run Plone 5.1.2 even out of the box as soon as plone.protect/utils.py is imported. And you are not the first trying to run it.
In the interest of research could you try to locate the python used for Plone, it should be something like /opt/plone/zeocluster/bin/python2.7 and start it then run the following code:

def test():
    def test2():
        return True
    d={"v":"myplone"}
    exec("print(v)",d)
test()

it is reproducing the code pattern that Python complains about in a very simplified way. It should run (and display 'myplone') without problem, if it displays the same error message there is something very wrong about your Plone python for your test machine.

Unrelated: I don't see why you have a zcml statement for collective.recaptcha in your buildout. It should run fine without it.

3

Many thanks for the reply.
Here's what I get...

peterf@emptiness-do05:~$ /opt/plone/zeocluster/bin/python2.7
Python 2.7.5+ (default, Feb 27 2014, 19:39:55) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def test():
...     def test2():
...         return True
...     d={"v":"myplone"}
...     exec("print(v)",d)
... test()
  File "<stdin>", line 6
    test()
       ^
SyntaxError: invalid syntax

In fact, the same thing happens with the system python!

peterf@emptiness-do05:~$ which python2.7
/usr/bin/python2.7
peterf@emptiness-do05:~$ /usr/bin/python2.7
Python 2.7.6 (default, Nov 23 2017, 15:50:55) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def test():
...     def test2():
...         return True
...     d={"v":"myplone"}
...     exec("print(v)",d)
... test()
  File "<stdin>", line 6
    test()
       ^
SyntaxError: invalid syntax
>>>

Thanks for the tip on the zcml, it's probably a remnant from some old recipe...

P

4

I forgot that you are not a python user, I expected you to save the text and run it from the command line like for example

python mytest.py

if you are entering directly the code with your fingers in the python interpreter, you need to define the test() function by typing return after the 'exec' line.

1 Like
5

Ooops :wink:
Nope, I'm a designer and Plone self-admin with barely enough knowledge to get in deep trouble on occasion!
Once I get pointed the right direction, I can usually make some progress, but the stupid newbie stuff gets me every time!

OK. This is with the system python.

    /usr/bin/python mytest.py 
      File "mytest.py", line 5
        exec("print(v)",d)
    SyntaxError: unqualified exec is not allowed in function 'test' it contains a nested function with free variables

and this with

/opt/plone/zeocluster/bin/python mytest.py 
  File "mytest.py", line 5
    exec("print(v)",d)
SyntaxError: unqualified exec is not allowed in function 'test' it contains a nested function with free variables

Hmmm.

$ /usr/bin/python
Python 2.7.6 (default, Nov 23 2017, 15:50:55) 
[GCC 4.8.4] on linux2

$ /opt/plone/zeocluster/bin/python 
Python 2.7.5+ (default, Feb 27 2014, 19:39:55) 
[GCC 4.8.1] on linux2

WTH??

Maybe a python reinstall is called for??
Best practice pointers??
Thanks so much, I'm way over my head here.
Fortunately, this is on a test droplet, I just wanted to update to 5.1.2 because I've had real problems with editing pages in mosaic in the RC...
Peter
@ideaswords

6

warning, updating system python yourself is not recommanded.

with my python 2.7.9 from ubuntu 16 there is no error at this point.
Is it possible to run python code on the net with different versions ? let's ask duckduckgo... (I know, I could install any python version with python/conda but it's tiresome)
trinket.io: needs registration: fail
skulpt.org: code runs but does not output anything: fail
IPython: I'm tired of this internet thingy, it was an interesting experiment but it's not really leading anywhere.

I have an old Ubuntu 12.04 LTS container, let's exhume it.

Ah! interesting ! Python 2.7.3 has the same behaviour as your oldish Pythons.

As a wild guess, I don't think that code testing with different Python versions include very old versions; testing with 2.7 is probably meaning 2.7.12. If Python has broken compatibilty somewhere, well, it's open source so upgrading is free. Not including time and effort to upgrade (but it's the same with software you are paying for)

IMO installing a new container is far easier than upgrading python, but as an Ubuntu user I'm biased.

Your pain comes from upgrading Plone code for Python 3 BTW. So many subtle problems...

7

in fact, this seems more like a bug introduced by @pbauer when trying to accomplish Python 3 compatibility:

please open an issue in plone.protect issue tracker.

in the mean time try pinning plone.protect = 3.1.2.

8

no need to, @pfraterdeus ! I have done it for you.

9

Hmm. I'm having a hard time getting my head around this.
If I'm testing my system python with the little test script from @Gp54321,
which fails in the command line, how does this plone.protect bug come into
the issue??
I have the same error after rolling back my test droplet (is that what you
mean by container?) to last month's backup...

So, reading today's notes, if I understand correctly, this is a bug in
older versions of python which only pops up due to the changes in currently
in 5.1.2?
So I should not be worrying about trying to fix Python on my system!?
Just pin plone.protect and carry on!
That would be a great help!

Many thanks!
Peter

10

that's exactly what I am saying. You can pin plone.protect although IMO the proper fix is to update your python since relying on old and unsupported software is not the best solution (You may take exception to that if your old python is supported by your distro)

11

And if the python in zeoinstance/bin/ is outdated when I'm trying to update
with buildout, is there a recipe to update python in buildout??

Many Thanks for your help, there be monsters here!

PF

12

adding a specific python version is done using a virtual environment; unfortunately there are too much riches here, there are a number of solutions. The only one I have used a bit is conda, but it's very much unloved so I'll not say more about the subject. Other people could advise you better on more pythonically correct ways (pyenv is one of the more used I think, if there is a forum on this tool it would be a good fit for this project)

13

OK, finally got this working on my test site. Thanks again for the help @gp54321
Unfortunately, the update didn't fix the problem I'm having with Mosaic refusing to Save edits.
I will post another query about this.