Not using bootstrap.py as default?

I started deleting bootstrap.py in all my buildouts. From my experience with current Plone 4.3 and 5.0 it is much more reliable this way.

  1. First unset setuptools and zc.buildout version pins in buildout.cfg:
    [versions] setuptools = zc.buildout =
  2. virtualenv .
  3. ./bin/pip install zc.buildout
  4. ./bin/buildout

This works much better for me. Advantages from my point of view:

  • no need to keep up to date with bootstrap.py (buildout breaks because of outdated version)
  • python way, pip is these days the de-facto standard. bootstrap.py was a workaround in pre-pip times.
  • requirements.txt possible if version pins are needed i.e. for deployment (i.e. buildout.coredev has it for Plone 5)

I'd like to see bootstrap.py disappear over time in favour of this approach.

Why don't we use this approach as default?

10 Likes

+1

I wrap this in a Make file :slight_smile:

2 Likes

:ok_hand:

I started using this approach too, especially when teaching Plone to new users.

1 Like

Nice, should go into docs!?

Looks good. What python do you use?
(I'm used to using one built by the Universal Installer)

Also, why do you not activate the virtualenv? (Just trying to understand here)

1 Like

./bin/pip integrates with Make :slight_smile:

1 Like

I never activated a virtualenv. This just leads to mistakes.

  • virtualenv installs a bin directory
  • with a python interpreter binary
  • with a pip already activated.
  • pip then installs buildout in bin and uses the already activated
    interpreter.
  • buildout then also uses the activated interpreter to install zope and all other scripts.
2 Likes

With the new developments in setuptools, I suggest pinning to a known working set for setuptools and zc.buildout. The pip install suggested does not guarantee success.

I've introduced the following combination for us at GW20E: zc.buildout=2.2.5 and setuptools=7.0
Also this works most of the time:

python bootstrap.py -c myfile.cfg --version=2.2.5

Combined with pinning:

setuptools = 7.0
zc.buildout = 2.2.5

The setuptools could probably be a bit newer, but I personally have no clue why they are developing setuptools at the rate they are. Only know it causes A LOT of problems which personally pisses me off. Spend so much time on issues related to this.

2 Likes

The whole point is: bootstrap.py is old, busted, no longer needed.

If you want to pin initial versions use a requirements.txt containing:

setuptools==17.11.1
zc.buildout==2.3.1

and doing a

./bin/pip install -r requirements.txt

This is the same as done with bootstrap --version in a much more reliable way.

Plone 4.3.x + 5.0x works here for me (and our environment at least) perfect with both - setuptools and zc.buildout - on current latest release. at time of writing this: setuptools 17.11.1 an zc.buildout 2.3.1

I do not see any reason to use older versions.

2 Likes

I'm all for dropping bootstrap.py, but only if the solution we provide and document is stable.
If you want to document, document something that works always, not just with the latest.

How many version of Plone are out there? How many specific workingsets?
How many 'not so default' versions of packages people might use? (ie.elementtree 1.2.7-20070827-preview).
Assuming that the latest version you suggested work for all those is assuming several bridges too far :wink:

Or you have to make a clear distinction between Plone > 4.3.X and < 4.3.X in documentation.

i fully agree. I'd introduce this officially with Plone 5 only, even if it works with 4.3.x. But with 5 its save to not break things accidentally I'd say.

I have been in setuptools/buildout hell many times, I can feel the pain and I would love to see a way out. With this proposal we are going to remove one of the components involved (bootstrap.py), which is good. Keeping bootstrap.py up-to-date is painful.

Though, we are not going to solve our setuptools/buildout problem with this, newer versions will still break things and I think there are quite a few disadvantages:

  1. We are going to introduce just another way of bootstrapping a buildout file. This will confuse people, since we will not be able to erase the old way of doing things in every single bit of our documentation. Some people started to use makefiles or even nix, which is even more confusing. I think we should only replace a solution if it offers a significant improvement.

  2. We will have a different approach for older Plone versions and Plone 5, this will confuse people.

  3. We are going to introduce a requirements.txt file that contains version pinning in addition to buildout.cfg/versions.cfg. We will have versions pinned in two different places. Instead of a bootstrap.py we will have a requirements.txt.

  4. The "buildout" command will not live in "bin/buildout" like all other buildout-like commands, this is confusing.

  5. Creating a virtualenv requires more typing than the two usual "bootstrap/buildout" steps. If you do this many times a day, this makes a difference (after trying the proposed approach for a few days this is actually my main concern :).

1 Like

ad 1) plone people are confused, python people are at home
ad 2) well, in Plone 2.5 we had no buildout at all, so change happens and is good if it improve things.
ad 3) indeed a problem. I'd love to see buildout using a requirements.txt instead of versions sections. this would unify things in python world a lot an reduce bloat of buildout coming from a time where pip just was not invented.
ad 4) wrong, its still in bin/buildout
ad 5) you can put in a bootstrap.sh script or (much better!) as @jaroel wrote just in a Makefile - this would unify things again, everybody in IT world knows makefiles (except some poor folks doing nothing else than Plone).

My far-away vision is to:

  • download a plone-5.0-requirements.txt
  • virtualenv .
  • ./bin/pip install -r plone-5.0-requirements.txt -r my-customer-specific-requirements.txt
  • ./bin/plone-create -c customer.cfg
  • ./bin/plone-customer start

or like so.

Just to get an idea what wrong with buildout in my opinion. Don't get me wrong, buildout is great (and I love this swiss knife), but way too much sometimes.

Re #4: it may be put in a location bin/buildout -- but I am under the impression that the virtualenv 'bin' dir is not always/often the same location as the 'bin' used in an application buildout?

All you need to know as a Plone developer right now is buildout:

All possible options discussed, introduce one or more additional concepts that people have to learn. Being more Pythonic is a worthwhile goal, and maybe even dropping buildout (though, I don't see how we could really replace that kind of functionality).

As long as we are not there, adding more concepts will just confuse people and at the same time not really improve the situation. The Python packaging hell will not freeze over just because we add a few more complexity layers. :wink:

Regarding bash scripts and make files: You are aware that you are proposing a solution to a problem that does not exist, if we stick with just buildout, right? Introducing a second layer of complexity because the first layer introduces a problem that would not exist without it, is never a good idea in my opinion.

1 Like

+100. couldn't agree more with @tisto

Seems to be the solution for now is to fix buildout rather than wrap it (for all the reasons @tisto mentions)

First: dont get the "far-away vision" too serious. I doubt we can implement it with our current stack. Maybe in Plone 10 :snail:

The major problem right now with buildout are version conflicts in several flavors. All the other stuff works great. So lets fix this and we have much less headache.

bootstrap.py is one source of such version conflicts, so getting rid of it first is a good start.

Next would be in my opinion to define the versions pip-style instead of in a version section. There are a bunch of implications then, so this need to be discussed in-depth and maybe I'am wrong.

The whole virtualenv can be put in an extra directory. I saw some folks doing a virtualenv ./venv or virtualenv ./python and so on. In that case all pip-installed scripts are living in ./venv/bin/* or ./python/bin/*.

I never do this and instead use virtualenv . (dot) - even if splitting up buildout generated scripts and virtualenv generated scripts is slightly cleaner.

On the other hand I found buildouts (in inherited customer projects) defining a custom bin-dir into the virtualenv - bin-dir in buildout is configurable too.

I prefer to have one central bin-dir for virtualenv and buildout together, even if there is a small possibility to override scripts installed by virtuenv with a buildout part generated script.