Thinking about a central way to handle Hotfixes

Plone Support Deployment & hosting
1

We prepared PloneHotfix20160419 on all of our buildouts,
while doing so we thought about a central way to handle hotfixes.

What do you guys think about a repo on github which contains a cfg with all current Hotfixes included.

Like https://raw.githubusercontent.com/plone/hotfixes/4.3.9/hotfixes.cfg

[buildout]
hotfix-eggs =
    # PloneHotfix20151006
    plone4.csrffixes

    Products.PloneHotfix20160419

[versions]
# PloneHotfix20151006
plone.protect = 3.0.17
plone.keyring = 3.0.1
plone.locking = 2.0.8
plone4.csrffixes = 1.0.9
cssselect = 0.9.1
plone4.csrffixes = 1.0.9  

Products.PloneHotfix20160419 = 1.0

Which you could extend by:

[buildout]
extends =
    http://dist.plone.org/release/4.3.9/versions.cfg
    https://raw.githubusercontent.com/plone/hotfixes/4.3.9/hotfixes.cfg
    base.cfg

[instance]
recipe = plone.recipe.zope2instance
eggs =
    Plone
    
   ${buildout:hotfix-eggs}

This would have spared me 6 hours of work to prepare all our buildouts.

2

In theory you need to maintain a hotfixes.cfg for each of the latest Plone versions - at least for 4.3-latest, 4.2-latest etc. Many sites however don't run on the latest 4.x-latest version. In addition the latest hotfixes have seen significant bugfixes the days after the release and in some case the latest version of a hotfix caused additional trouble instead of solving problems. In this case we had to pin certain versions which did not represent the latest version of e.g. plone4.csrffixes.

-aj

3

Anyone else interested in this?

4

There is http://jone.github.io/plone-hotfixes/ as well.

5

sounds like a good idea.

6

well first of all this already exists in some way: https://plone.org/security/hotfix_json

I started once an addon buildout.autoapplyplonehotfixes which reads this json and apply the necessary Hotfixes
It is based on https://github.com/plone/plone.vulnerabilitychecks.core

Any comments and help is welcome.

The JSON has one Problem with the plone4.csrffixes, as this Hotfix did not meet the Naming Guidelines for PloneHotfixes

7

So you need a manual exception for "plone4.csrffixes". else this way to handle hotfixes sounds not bad.

A full Hotfix session took us 15 Man-hours, so we defenitely will go for a hotfixes repo at github next time.

8

Sorry to get late on the conversation... I saw others saying as well "it took us XX man hours to push" so spending some extra XX man hours on making that process even faster for everyone would be extra extra cool :wink:

9

If anyone is interested this is a working proof of concept:

2 Likes
10

While the starzel buildout is nice, i would love to have this hotfixes in collective, you might want to go for Products.CMFPlone instead of Plone but i'm not sure about that.

I like what you did, simple and sure working.

11

Still this is a proof of concept!

Also it would be nice to read a json.

I hope that for the next hotfix we will have a proper solution :wink:

12

Why a JSON, not sure i understand that.

13

To avoid fiddling with strings, which is error prone!
it is better to have something that works like:

options = loads(urlopen(base_url % version).read)
return option['eggs']

or something similar (you may want to have also options['versions']).

Indeed in a recipe you can do a lot more...

14

I see, JSON seems a good choice for that.

Lets talk with others to get a "hotfixes" repo in collective, then maintain those extendable (with .cfg) and as JSON.

I think about this layout:

Extendable for none mr.scripty users:

Contains all hotfixes without plone4.csrffixes

https://raw.githubusercontent.com/collective/hotfixes/buildout/normal/<version>.cfg

Includes plone4.csrffixes and plone.protect pins

https://raw.githubusercontent.com/collective/hotfixes/buildout/extended/<version>.cfg

JSON for mr.scripty users:

Contains all hotfixes without plone4.csrffixes

https://raw.githubusercontent.com/collective/hotfixes/json/normal.json

Includes plone4.csrffixes and plone.protect pins

https://raw.githubusercontent.com/collective/hotfixes/json/extended.json

I'll write ./generator.py which will transform the JSON to extendable by buildout files.

15

I use a products directory to install hotfixes rather than run buildout. For many older setups that might have old weird stuff in their buildouts, it just introduces more risk than I'd like to rerun buildout for a patch.

In an ideal world, it would be nice to go to a site setup, and click "apply hotfixes" and sit back and it happens.

I think it might be possible. Writing a view to download from a known source and unpack a zip into a products dir from within plone is not hard. Doing it such that zeo clients that, don't know about each other, do so one at a time, in a way that they can roll themselves back in the case of failure to start... interesting problem to solve.

Perhaps some zodb object that registers IP and dir for each zope instance that starts. Then a clock tick on each that works out if its this instance turn to download and restart?

1 Like
16

Thanks for your input @djay.

I had the idea of Web-installable Products at the PloneConf 2015. After talking with Philipp about that I knew that this will be just a security risk.

Also having to login to a lots of sites and clicking that update button is more work than running ./bin/buildout and restart the instances.

17

@pcdummy. It's a calculated risk. Given it would be a fixed source, possibly fixed to an IP address the risk is moderate. And it could be explicitly disabled for those not willing to take the risk.

Having to login to lots of sites is much easier for those without much technical knowledge, yet they want to keep their site secure. For lots of sites, there is no one with enough experience left to run buildout, and deal with any failures.

We should be making plone "expensive consultant" proof.

18

cryptographic signing would further reduce the risk.

19

We at the Webmeisterei have been talking about that, we will implement a buildout recipe: plone.recipe.hotfixes

It will do automatic version detection and will have the possiblity to give a a base_url where you have one or many JSON files hosted. We will create and maintain the JSON files with the recipe itself on: https://github.com/collective/plone.recipe.hotfixes

The schema of the JSON files is not yet finished, but we will implement a way that you can include/exclude certain Hotfix (like the CSRF Fixes) by giving/not giving that config as parameter.

Here an example:

[buildout]
eggs +=
    ${hotfixes:eggs}

versions +=
    ${hotfixes:versions}

[versions]
Products.CMFPlone = 4.3.11

[hotfixes]
recipe = plone.recipe.hotfixes
base_url = https://raw.githubusercontent.com/collective/plone.recipe.hotfixes/hotfixes/
configs =
    base
    csrf

The example above will install all 4.3.11 hotfixes including plone4.csrffixes

Please let me know if you have any ideas to improve this!

3 Likes
20

Sweet!