Buildout - unnecessary uninstall?

Plone Support
1

I originally posted an issue on the plone.recipe.haproxy github, but I think this is actually probably an issue with buildout. I'm using zc.buildout 2.5.2 for reference. The original post I made is quoted below. Anyway, looking into it a bit more I think it's a problem with buildout because the

My problem is that I will sometimes run buildout to update or add a python egg, but even though I have changed nothing with my haproxy part configs it will try to rebuild (why?). This in itself isn't so bad, but the rebuild process tries to delete ./parts/haproxy-build/sbin, where there is an NFS file lock because the process is running. I can't delete this and thus rebuild until I stop the running process, so now I have a few minutes downtime on my site to rebuild haproxy. Not good!

It doesn't matter that I run this with ${buildout:location}/bin/haproxy because that is just a wrapper for running the file in sbin.

As you can see it's trying to uninstall:
Uninstalling haproxy-build.
While:
Installing.
Uninstalling haproxy-build.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1992, in main
    getattr(buildout, command)(args)
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/buildout.py", line 605, in install
    self._uninstall_part(part, installed_part_options)
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/buildout.py", line 731, in _uninstall_part
    installed_part_options[part]['__buildout_installed__'])
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/buildout.py", line 840, in _uninstall
    rmtree(f)
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/rmtree.py", line 60, in rmtree
    shutil.rmtree (path, onerror = retry_writeable)
  File "/usr/local/python-2.7.10/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/local/python-2.7.10/lib/python2.7/shutil.py", line 256, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "[redacted]/lib/python2.7/site-packages/zc/buildout/rmtree.py", line 58, in retry_writeable
    func (path)
OSError: [Errno 39] Directory not empty: '[redacted]/parts/haproxy-build/sbin'

I just don't get why. I didn't change anything about this, in fact all I did was update a pinned version number of some python egg. Are there any known bugs with buildout being overly aggressive with uninstalls? My plan is to take the haproxy parts out of my main buildout (my buildout.cfg extends haproxy.cfg where those parts are defined) at least.

2

I always wondered why people for a time were using buildout to set up everything in their Plone stack. In your place I would definitely stop having buildout provide haproxy and would provide haproxy through system packages e.g. apt-get

3

I blame Martin Aspeli

2 Likes
4

This is generally safe :slight_smile:

5

+1 for not installing haproxy via buildout.
but buildout can be handy for providing configuration files for haproxy.
this is what I often do.

however, buildout just runs everything again. if you have only changed a specific part, you can also do:

1 Like
6

I've done that before but lately that has led to other problems when using filestorage and backup recipes - I'd end up needing to run all of those parts as well as the parts for my ZEO clients, or the needed dbs to mount would be missing, or backups would fail, etc. Being able to trust my configuration instead of remembering all things to do within my brain was thus appealing.

To be honest, we were already in the process of moving the haproxy setup outside of our main buildout for this reason. I was prompted to make this topic because I thought perhaps the real problem was something wrong with my settings. It sounds like that is not the case and we do just want to keep them separate, so that is good to know.

7

When I remember right, the decision between "update" versus "uninstall/install" is taken by the recipe (---> "buildout" documentation). Likely, you must look at the "haproxy" recipe code to find out why it decides for "uninstall+install".

Like all software components, "buildout" make some assumptions. One of those assumptions is that anything below parts can be removed and rebuilt (e.g. during "uninstall/install"). Therefore, no runtime file or long lasting resource file should reside there. For this reason, the base recipes puts those files below "var" (and not below "parts"). The "haprox" recipy should do the same.