I started using this approach too, especially when teaching Plone to new users.
I started using this approach too, especially when teaching Plone to new users.
Nice, should go into docs!?
Looks good. What python do you use?
(I'm used to using one built by the Universal Installer)
Also, why do you not
activate the virtualenv? (Just trying to understand here)
./bin/pip integrates with Make
I never activated a virtualenv. This just leads to mistakes.
With the new developments in setuptools, I suggest pinning to a known working set for setuptools and zc.buildout. The pip install suggested does not guarantee success.
I've introduced the following combination for us at GW20E: zc.buildout=2.2.5 and setuptools=7.0
Also this works most of the time:
python bootstrap.py -c myfile.cfg --version=2.2.5
Combined with pinning:
setuptools = 7.0
zc.buildout = 2.2.5
The setuptools could probably be a bit newer, but I personally have no clue why they are developing setuptools at the rate they are. Only know it causes A LOT of problems which personally pisses me off. Spend so much time on issues related to this.
The whole point is: bootstrap.py is old, busted, no longer needed.
If you want to pin initial versions use a requirements.txt containing:
and doing a
./bin/pip install -r requirements.txt
This is the same as done with bootstrap --version in a much more reliable way.
Plone 4.3.x + 5.0x works here for me (and our environment at least) perfect with both - setuptools and zc.buildout - on current latest release. at time of writing this: setuptools 17.11.1 an zc.buildout 2.3.1
I do not see any reason to use older versions.
I'm all for dropping bootstrap.py, but only if the solution we provide and document is stable.
If you want to document, document something that works always, not just with the latest.
How many version of Plone are out there? How many specific workingsets?
How many 'not so default' versions of packages people might use? (ie.elementtree 1.2.7-20070827-preview).
Assuming that the latest version you suggested work for all those is assuming several bridges too far
Or you have to make a clear distinction between Plone > 4.3.X and < 4.3.X in documentation.
i fully agree. I'd introduce this officially with Plone 5 only, even if it works with 4.3.x. But with 5 its save to not break things accidentally I'd say.
I have been in setuptools/buildout hell many times, I can feel the pain and I would love to see a way out. With this proposal we are going to remove one of the components involved (bootstrap.py), which is good. Keeping bootstrap.py up-to-date is painful.
Though, we are not going to solve our setuptools/buildout problem with this, newer versions will still break things and I think there are quite a few disadvantages:
We are going to introduce just another way of bootstrapping a buildout file. This will confuse people, since we will not be able to erase the old way of doing things in every single bit of our documentation. Some people started to use makefiles or even nix, which is even more confusing. I think we should only replace a solution if it offers a significant improvement.
We will have a different approach for older Plone versions and Plone 5, this will confuse people.
We are going to introduce a requirements.txt file that contains version pinning in addition to buildout.cfg/versions.cfg. We will have versions pinned in two different places. Instead of a bootstrap.py we will have a requirements.txt.
The "buildout" command will not live in "bin/buildout" like all other buildout-like commands, this is confusing.
Creating a virtualenv requires more typing than the two usual "bootstrap/buildout" steps. If you do this many times a day, this makes a difference (after trying the proposed approach for a few days this is actually my main concern :).
ad 1) plone people are confused, python people are at home
ad 2) well, in Plone 2.5 we had no buildout at all, so change happens and is good if it improve things.
ad 3) indeed a problem. I'd love to see buildout using a requirements.txt instead of versions sections. this would unify things in python world a lot an reduce bloat of buildout coming from a time where pip just was not invented.
ad 4) wrong, its still in bin/buildout
ad 5) you can put in a bootstrap.sh script or (much better!) as @jaroel wrote just in a Makefile - this would unify things again, everybody in IT world knows makefiles (except some poor folks doing nothing else than Plone).
My far-away vision is to:
or like so.
Just to get an idea what wrong with buildout in my opinion. Don't get me wrong, buildout is great (and I love this swiss knife), but way too much sometimes.
Re #4: it may be put in a location bin/buildout -- but I am under the impression that the virtualenv 'bin' dir is not always/often the same location as the 'bin' used in an application buildout?
All you need to know as a Plone developer right now is buildout:
All possible options discussed, introduce one or more additional concepts that people have to learn. Being more Pythonic is a worthwhile goal, and maybe even dropping buildout (though, I don't see how we could really replace that kind of functionality).
As long as we are not there, adding more concepts will just confuse people and at the same time not really improve the situation. The Python packaging hell will not freeze over just because we add a few more complexity layers.
Regarding bash scripts and make files: You are aware that you are proposing a solution to a problem that does not exist, if we stick with just buildout, right? Introducing a second layer of complexity because the first layer introduces a problem that would not exist without it, is never a good idea in my opinion.
+100. couldn't agree more with @tisto
Seems to be the solution for now is to fix buildout rather than wrap it (for all the reasons @tisto mentions)
First: dont get the "far-away vision" too serious. I doubt we can implement it with our current stack. Maybe in Plone 10
The major problem right now with buildout are version conflicts in several flavors. All the other stuff works great. So lets fix this and we have much less headache.
bootstrap.py is one source of such version conflicts, so getting rid of it first is a good start.
Next would be in my opinion to define the versions pip-style instead of in a version section. There are a bunch of implications then, so this need to be discussed in-depth and maybe I'am wrong.
The whole virtualenv can be put in an extra directory. I saw some folks doing a
virtualenv ./venv or
virtualenv ./python and so on. In that case all pip-installed scripts are living in
I never do this and instead use
virtualenv . (dot) - even if splitting up buildout generated scripts and virtualenv generated scripts is slightly cleaner.
On the other hand I found buildouts (in inherited customer projects) defining a custom bin-dir into the virtualenv - bin-dir in buildout is configurable too.
I prefer to have one central bin-dir for virtualenv and buildout together, even if there is a small possibility to override scripts installed by virtuenv with a buildout part generated script.
Let me note that we already have nearly all the problems that have been mentioned as a negative for replacing bootstrap.py. Everyone that is checking out an old buildout or trying to update an old site is running into the fact that bootstrap.py is broken without extraordinary workarounds.
So, we already need to do some documentation to help these poor souls out of their bootstrap/setuptools hell. At the moment, it looks easiest to me to document the virtualenv/buildout approach.
I've just found time to come back & read this. I want to say thanks to Jens - for the virtualenv idea & everyone for the very helpful open discussion. I think the best thing that can be put in the docs is a clear admission that this area gets difficult at times... that should keep new adopters with us, rather than giving up!
Btw yesterday I found the following command which seems like it would help with version conflicts when I get them (is this obvious, or not?) In this example for plone.resource, it shows me what other products require it & what pins they apply:
$ grep plone.resource eggs/*.egg/EGG-INFO/requires.txt eggs/plone.app.theming-1.1.6-py2.7.egg/EGG-INFO/requires.txt:plone.resource>=1.0b5 eggs/plone.app.theming-1.1.6-py2.7.egg/EGG-INFO/requires.txt:plone.resourceeditor eggs/plone.app.themingplugins-1.0b1-py2.7.egg/EGG-INFO/requires.txt:plone.resource eggs/plone.resourceeditor-1.0-py2.7.egg/EGG-INFO/requires.txt:plone.resource
Worked fine for a while but ...
Plone Foundation Code of Conduct