Installation trouble with 5.2.14

djr · April 19, 2024, 2:16pm

I'm having an installation problem with a Plone server that I inherited. After a recent round of server updates, one of the Plone 5.2.1 sites won't start. This particular site uses Python3, and it has symbolic links to Python 3.6 libraries in /usr/lib/python3.6. It looks like one of the server updates replaced 3.6 with 3.8, so the sym links in the Plone tree are broken.

Is there a way to fix the installation as it is?

If not, I have tried installing Plone 5.2.14, but the install script fails with:

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/opt/plone5-j2/zeocluster/lib/python3.8/site-packages/zc/buildout/buildout.py", line 2252, in main
getattr(buildout, command)(args)
[...]
File "/opt/plone5-j2/zeocluster/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 196, in load
return functools.reduce(getattr, attrs, module)
AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
Buildout failed. Unable to continue

What am I missing with this?

Thanks for any advice.

yurj · April 19, 2024, 9:32pm

did you try to simlink /usr/lib/python3.6 to /usr/lib/python3.8? Anyway, this should be the fix

it should be a problem with the setuptools version.

djr · April 22, 2024, 1:23pm

I tried the sym link, but that didn't work. I can try again, along with recreating venv.

djr · April 22, 2024, 4:21pm

No luck with that:

pip3 install -r requirements.txt -c constraints.txt setuptools zc.buildout wheel

Requirement already satisfied: setuptools==65.7.0 in /usr/local/lib/python3.8/dist-packages (from -c constraints.txt (line 2)) (65.7.0)
Requirement already satisfied: zc.buildout==3.0.1 in /usr/local/lib/python3.8/dist-packages (from -c constraints.txt (line 3)) (3.0.1)
Requirement already satisfied: wheel==0.38.4 in /usr/local/lib/python3.8/dist-packages (from -c constraints.txt (line 4)) (0.38.4)
Collecting pip==23.0.1
Using cached pip-23.0.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'pip'. No files were found to uninstall.
Successfully installed pip-23.0.1
root@host:/opt/plone5-j/zeocluster#
root@host:/opt/plone5-j/zeocluster# ./bin/buildout
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "/opt/plone5-j/zeocluster/lib/python3.6/abc.py", line 64, in
ModuleNotFoundError: No module named '_abc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/plone5-j/zeocluster/lib/python3.6/io.py", line 52, in
File "/opt/plone5-j/zeocluster/lib/python3.6/abc.py", line 68, in
ModuleNotFoundError: No module named '_py_abc'
Aborted (core dumped)

seanupton · April 22, 2024, 5:13pm

Check that the Python 3.8 you are using is not broken in this regard, maybe check:

$ python3
Python 3.8.18 (default, Feb 18 2024, 11:46:52)
[Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
>>> import _abc
>>> _abc
<module '_abc' (built-in)>

You have not indicated the server/system operating environment you are using? It is possible that your system Python might just be broken and/or incomplete (more likely latter, as a result of distro packaging concerns). I might advise:

(as the user you run Plone as) install pyenv via download of pyenv-installer
use Pyenv to install Python 3.8.18 from source and configure your bash profile/rc accordingly.
set pyenv local 3.8.18 in the directory you plan to create a venv
create a venv via python3.8 -m venv env in that directory
create a venv with reasonable requirements for Plone 5.2.14, make a requirements.txt that looks like:

setuptools==65.7.0
pip==20.3.4
wheel==0.38.4
zc.buildout==3.0.1
Pillow==9.2.0
lxml==4.8.0

Use your virtual env python to install these, something like:

./env/bin/python -m pip install -r requirements.txt

Use the buildout in your environment to run buildout, something like:

./env/bin/buildout -c buildout.cfg

Notes and Disclaimers:

This is all speculative, you may be able to make your system Python to play nice, but I don't know what the sufficient conditions for that are.
Make a clone or copy of your server if you can instead of experimenting in place. If you cannot do that, have good backups before trying to make changes, such that you can roll back to wherever you were before experimenting.
My recollection is that there are a bunch of YouTube videos on installing Python via pyenv, may be worth finding one or two that walk through this. This path ends up being much more "in your sphere of control" than a system Python ever could be (both on Linux and macOS).

djr · April 22, 2024, 9:20pm

This is all on a server running Ubuntu 20.04.

I installed Python 3.8.18 from source into /usr/local. I then tried installing Plone 5.2.14 again, picking /usr/local/bin/python3.8 as the version of Python to use. It gets a lot farther, but still fails with the following:

chmod 754 /opt/plone5-j2/zeocluster/bin/*'
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python': Operation not permitted
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python3': Operation not permitted
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python3.8': Operation not permitted
Section setpermissions contains unused option(s): 'update-command'.
This may be an indication for either a typo in the option's name or a bug in the used recipe.
Installing precompiler.
Compiling Python files.
File "/opt/plone5-j2/buildout-cache/eggs/cp38/Zope-4.8.10-py3.8.egg/App/tests/fixtures/error.py", line 2
a = # noqa
^
SyntaxError: invalid syntax

File "/opt/plone5-j2/buildout-cache/eggs/cp38/plone.staticresources-1.4.6-py3.8.egg/plone/staticresources/static/components/jqtree/_entries/insert.py", line 44
print 'rename %s to %s' % (old_filename, new_filename)
^
SyntaxError: invalid syntax

File "/opt/plone5-j2/buildout-cache/eggs/cp38/plone.staticresources-1.4.6-py3.8.egg/plone/staticresources/static/components/jqtree/_entries/renumber.py", line 36
print 'rename %s to %s' % (old_filename, new_filename)
^
SyntaxError: invalid syntax

File "/opt/plone5-j2/buildout-cache/eggs/cp38/zodbpickle-2.6-py3.8-linux-x86_64.egg/zodbpickle/pickletools_2.py", line 1803
print "skipping %r: it doesn't look like an opcode name" % name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("skipping %r: it doesn't look like an opcode name" % name)?

File "/opt/plone5-j2/buildout-cache/eggs/cp38/zodbpickle-2.6-py3.8-linux-x86_64.egg/zodbpickle/pickle_2.py", line 882
except _Stop, stopinst:
^
SyntaxError: invalid syntax

File "/opt/plone5-j2/buildout-cache/eggs/cp38/zodbpickle-2.6-py3.8-linux-x86_64.egg/zodbpickle/tests/pickletester_2.py", line 433
x = [0, 1L, 2.0, 3.0+0j]
^
SyntaxError: invalid syntax

Compiling locale files.
/opt/plone5-j2/buildout-cache/eggs/cp38/plone.app.contenttypes-2.2.3-py3.8.egg/plone/app/contenttypes/migration/utils.py:504: SyntaxWarning: "is" with a literal. Did you mean "=="?
if source_type is 'DX':
/opt/plone5-j2/buildout-cache/eggs/cp38/plone.app.contenttypes-2.2.3-py3.8.egg/plone/app/contenttypes/migration/utils.py:507: SyntaxWarning: "is" with a literal. Did you mean "=="?
if target_type is 'AT' and not is_referenceable(source_obj):
Buildout failed. Unable to continue

Installation has failed.

dieter · April 23, 2024, 5:00am

Don Rainwater via Plone Community wrote at 2024-4-22 21:30 +0000:

This is all on a server running Ubuntu 20.04.
...

I installed Python 3.8.18 from source into /usr/local. I then tried installing Plone 5.2.14 again, picking /usr/local/bin/python3.8 as the version of Python to use. It gets a lot farther, but still fails with the following:

chmod 754 /opt/plone5-j2/zeocluster/bin/*'
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python': Operation not permitted
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python3': Operation not permitted
chmod: changing permissions of '/opt/plone5-j2/zeocluster/bin/python3.8': Operation not permitted
Section setpermissions contains unused option(s): 'update-command'.
This may be an indication for either a typo in the option's name or a bug in the used recipe.
Installing precompiler.
Compiling Python files.
File "/opt/plone5-j2/buildout-cache/eggs/cp38/Zope-4.8.10-py3.8.egg/App/tests/fixtures/error.py", line 2
a = # noqa
^
SyntaxError: invalid syntax

This is by purpose!

...
File "/opt/plone5-j2/buildout-cache/eggs/cp38/plone.staticresources-1.4.6-py3.8.egg/plone/staticresources/static/components/jqtree/_entries/insert.py", line 44
print 'rename %s to %s' % (old_filename, new_filename)
^
SyntaxError: invalid syntax

Apparently, Python 3 tries to compile Python 2 code.

I do not know how you arrived there. At your place, I would check
where the version spec ("1.4.6") comes from (you may try buildout annotate)
and then replace it with a Python 3 compatible version.

yurj · April 23, 2024, 7:09am

there are python 2 py but this is not the problem, they're just ignored. @djr there should be a install log somewhere with the real cause.

I would use a python 3.6. Check also the requirements for Plone to build a suitable python.

djr · April 23, 2024, 12:34pm

To me, it doesn't look like the install.log file gives much info as to why it is failing:

/usr/local/src/Plone-5.2.14-UnifiedInstaller-1.0/install.log:

Detailed installation log
Starting at Mon 22 Apr 2024 04:43:54 PM EDT
Creating Python virtual environment.
Virtualenv successfully created
Installing Python requirements in virtual environment.

WARNING: The directory '/bin/false/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.

Ignoring setuptools: markers 'python_version < "3.0"' don't match your environment
Ignoring zc.buildout: markers 'python_version < "3.0"' don't match your environment
Ignoring wheel: markers 'python_version < "3.0"' don't match your environment
Ignoring pywin32: markers 'platform_system == "Windows"' don't match your environment
Ignoring certifi: markers 'platform_system == "Windows"' don't match your environment
Ignoring ntfsutils: markers 'platform_system == "Windows" and python_version < "3.0"' don't match your environment
Collecting setuptools==65.7.0
Downloading setuptools-65.7.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 2.0 MB/s eta 0:00:00
Collecting zc.buildout==3.0.1
Downloading zc.buildout-3.0.1-py2.py3-none-any.whl (168 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.4/168.4 kB 2.2 MB/s eta 0:00:00
Collecting wheel==0.38.4
Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
Collecting pip==20.3.4
Downloading pip-20.3.4-py2.py3-none-any.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 2.5 MB/s eta 0:00:00
Collecting Pillow==9.2.0
Downloading Pillow-9.2.0-cp38-cp38-manylinux_2_28_x86_64.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 3.3 MB/s eta 0:00:00
Collecting lxml==4.8.0
Downloading lxml-4.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 2.1 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip, Pillow, lxml, zc.buildout
Attempting uninstall: setuptools
Found existing installation: setuptools 56.0.0
Uninstalling setuptools-56.0.0:
Successfully uninstalled setuptools-56.0.0
Attempting uninstall: pip
Found existing installation: pip 23.0.1
Uninstalling pip-23.0.1:
Successfully uninstalled pip-23.0.1
Successfully installed Pillow-9.2.0 lxml-4.8.0 pip-20.3.4 setuptools-65.7.0 wheel-0.38.4 zc.buildout-3.0.1
Create buildout: cluster
Copying buildout skeleton to /opt/plone5-j2/zeocluster
Building Zope/Plone; this takes a while...
Parts of the install are missing: path '/opt/plone5-j2/zeocluster/parts/client2'. Buildout must have failed. Aborting.

Instance directory /opt/plone5-j2/zeocluster contains:

requirements.txt
develop-eggs
README.html
adminPassword.txt
lxml_static.cfg
.installed.cfg
parts
include
bin
buildout.cfg
var
lib64
lib
develop.cfg
base.cfg
pyvenv.cfg
products

Instance bin directory /opt/plone5-j2/zeocluster/bin contains:

zconsole
pip3
zeopack
client1
Activate.ps1
pip3.8
snapshotbackup
runwsgi
restore
python3
zeoserver
zopepy
activate.csh
pip
mkwsgiinstance
backup
activate
python3.8
wheel
buildout
repozo
plonectl
python
snapshotrestore
addzope2user
activate.fish

Instance parts directory contains:

client1
README.txt
zeoserver

Buildout failed. Unable to continue

yurj · April 23, 2024, 3:09pm

github.com

plone/Installers-UnifiedInstaller/blob/b5970f4fab712b6d16a64da946e904da676149e7/helper_scripts/create_instance.py#L277-L292


      
          if opt.itype == "standalone":
              paths_to_check += [
                  os.path.join(opt.instance_home, "parts", "instance"),
              ]
          else:
              paths_to_check += [
                  os.path.join(opt.instance_home, "bin", "zeoserver" + ext),
                  os.path.join(opt.instance_home, "parts", "zeoserver"),
                  os.path.join(opt.instance_home, "parts", "client1"),
                  os.path.join(opt.instance_home, "parts", "client2"),
              ]
          for path_to_check in paths_to_check:
              if not os.path.exists(path_to_check):
                  _print(
                      "Parts of the install are missing: path '{}'. Buildout must have failed. Aborting.".format(
                          path_to_check

It seems it cannot create the client2 directory. Maybe some permission problem? The original installation had client2?

djr · April 23, 2024, 3:32pm

Why would that happen? The target installation directory (/opt/plone5-j2) doesn't exist when I start, and the installer is able to create everything else in that tree. I'm also running the installer as root, so access/permissions shouldn't be an issue.

yurj · April 24, 2024, 7:54am

Free space on disk, for example. Anyway, if you run
./install.sh --help you can see some other options. For example, you can use --clients=1 and see what happen. You can also put a breakpoint() in helper_scripts/create_instance.py and see why it fails to create the second instance. At line 939 of helper_scripts/main_install_script.sh there's the instance creation.