I admit, I did some merging of buildout.cfg / versions.cfg code in my project, and I'm currently to a bootstrap.py-less buildout layout; but more informative error message would have been nice ...
When I try to build my instance, I get the following error:
Generated interpreter '/opt/zope/instances/therp-shop3/parts/instance/bin/interpreter'.
While:
Installing instance.
An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/opt/zope/instances/therp-shop3/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
getattr(buildout, command)(args)
File "/opt/zope/instances/therp-shop3/lib/python2.7/site-packages/zc/buildout/buildout.py", line 797, in install
installed_files = self[part]._call(recipe.install)
File "/opt/zope/instances/therp-shop3/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1557, in _call
return f()
File "/opt/zope/common/eggs/plone.recipe.zope2instance-4.4.0-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 142, in install
self.build_package_includes()
File "/opt/zope/common/eggs/plone.recipe.zope2instance-4.4.0-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 780, in build_package_includes
raise ValueError('Invalid zcml', orig)
ValueError: ('Invalid zcml', '#')
I found the error to be caused by a package value of '#' (doesn't look like zcml to me, by the way) and changed my /opt/zope/common/eggs/plone.recipe.zope2instance-4.4.0-py2.7.egg/plone/recipe/zope2instance/__init__.py like so:
if zcml: # this is line 757
n = 1 # 001 is reserved for an optional locales-configure
package_match = re.compile('\w+([.]\w+)*$').match
for package in zcml:
if package == '#': # HOTFIX
continue # HOTFIX
n += 1
orig = package
...
Now, should the recipe ignore comments somewhere, or is there an error in one of my packages? And how can I find out, which one?
(Plone 4.3.3 - currently under reconstruction to make it updateable; zc.buildout 2.11.4 or 2.11.0; setuptools 39.2.0; plone.recipe.zope2instance v4.4.0 because of the new setuptools)
With which python version are you running your buildout? From the information in your question I get the hunch this might be more related to a Python language imcompatibility than a Plone/Zope programming error.
I don't know what your own experience with Plone/Zope development is, but here is some general advice on changing version pinnings for others in the community reading this. I see some large version differences in your question: You're pinning c.r.zope2instance to the latest available 4.4.0 from 2018, but your Plone instance is running 4.3.3 with most of its pinnings still from around March 2014.
Plone is composed of a lot of different packages and with every new release, be it major or minor, a new 'known good set' (KGS) is created by the release manager and other community members. This is then released as "Plone 4.3.3" or 'Plone 4.3.17'. You can find those releases in its 'source' on https://dist.plone.org/release/ , the main entry file is the versions.cfg in every 'version' directory.
If you are really experienced with Plone development and maintenance, follow up which bugs get fixed in which packages and know the impact of those changes, you can cherry pick packages. But at the time of release these packages were tested and worked with the released Plone 'known good set' in that period.
The recent update to new pypi infrastructure and pypi.org, the move to https and changes for that in setuptools have been causing some issues with existing Plone deployments, mainly in the packaging parts. For older instances that haven't been kept up to date (or only had the security Patches), Patching the setuptools/zc.buildout blocks and then patching every next issue you run into can take a lot of time.
If you don't have too much custom development in your site, keeping up to date once a year with minor Plone releases is almost always the preferred balance between effort spent and the number of minor changes 'forced' upon you that you have to handle.
thank you for your thorough answer. To answer your questions:
Python 2.7.5 (System package 2.7.5-39.el7_2 on CentOS 7, to be exact; about to be updated)
buildout version 2.11.4
pip 10.0.1 from /opt/zope/instances/my-instance/lib/python2.7/site-packages/pip (python 2.7)
I have inherited a quite heavily customized Plone project with a bunch of undocumented version pinnings which mostly are those of Plone 4.3.3 (some newer ones as well); it was based on Plone 3 originally, and the developer who switched it to buildout and Plone 4 said, "Plone doesn't want to be updated", and thus left to me a mess. I'm currently working to restructure it to make it testable and updateable.
Yes, you're right, there is that large version difference: The build tools are much newer than the Plone packages. I assumed this to be worth a try, because I had much "fun" trying to fix version conflicts with the old toolset, and there are big improvements in the newer versions. The main gotcha is that newer buildout doesn't tolerate "plone.package >= 1.2.3" version "pinnings" in versions.cfg anymore, but this can be fixed easily; I'm able to build.
I would be much more hesitant to build new packages with an old toolset than this way around.
I had a look at the change history of plone.recipe.zope2instance and couldn't find any reason why version 4.4.0 shouldn't work with Plone 4.3.3; Plone 5 is targeted by version 5+. And indeed, it seems to work; there is just that bad error handling. I created an issue for the recipe, and I might create a pull request as well, but I'm not experienced at all with recipe development.