Renaming a policy package

Plone Support Add-on Development
,
1

I'm working on upgrading the site of one of our customers; currently, we are using an old pattern consisting on 3 different packages: buildout, policy and theme. this pattern has proven complex and hard to maintain for obvious reasons and we want to migrate it to a single package including the whole code.

let's summarize that as follows:

  • current: example.buildout, example.policy and example.theme
  • new: example.portal

the problem I'm facing right now is that I just realized, after having moved almost all code to this unified new package, that I will have a huge issue regarding the policy and theme package being installed; let me explain better:

my original idea was just to copy the ZODB and blob storage, and drop them in the location of my new unified package; obviously I will have to deal with persistent objects pointing to the old code so I was planning to do that adding something like this to my __init__.py module:

import sys
sys.modules['example.policy'] = sys.modules['example.theme'] = sys.modules[__name__]

note that all previous code will be available in the same locations so, if I was previously importing something from example.policy.caching, it will be available now from example.portal.caching; similarly, if I was previously importing something from example.theme.browser, it will be available now from example.portal.browser.

so far, so good; but then comes the hardest problem: if I just move this way, I will be left with 2 packages marked as installed and they will be no longer there; also, my new package will not be marked as installed and I will be left with some issues as I don't want to reinstall it because all the stuff being in the default profile is already persisting in the ZODB.

I was digging the code of the QuickInstallerTool trying to figure out if it is possible just to delete the pid form it or not:

seems something doable, are there any risks?

what about marking a package as installed without actually installing it? is that possible? the code there is out of my comprehension.

should I just remove all the XML files out of the default profile, install the package and move on?

2

https://pypi.python.org/pypi/zodbupdate is your friend for existing objects.
Add deprecation warning at the old location for code that imports from there.

Not sure on the GenericSetup profiles, though. This sounds like something that @mauritsvanrees would know.

-Roel

3

thanks! after thinking a little bit more on this, I'm going to follow a different approach: my new example.portal package will depend on the old code and I will move/remove stuff slowly, one piece at a time, until I can just get rid of the old packages by uninstalling them.

the whole thing is quite complex and I think is better to be evolutionary than revolutionary.

4

I share this pain. We (also) adopted this from "Professional
Plone Development", but have had the same experience of burden to maintain. I wonder, how other have managed this.

Our specialty is that we have a lot of very similar sites. At first, I begun to merge themes with policy packages. Then we built a "configuration application" to compose out buildouts from shared parts, but kept tracking the composed single file buildout with buildout repository (it was essential to get rid of custom buildout extends to really track the changes). Next step was making a one multi-profile policy package. It really speed up getting same customizations (like ZCA overrides / layers) to multiple sites safely. And it's easier to add upgrade steps per feature. Finally, with Plone 5 we moved to zip-uploaded fat themes to deliver theme related updates (well, also a lot of registry.xml) without restart.

5

One alternative way of solving this, what we did in ploneintranet: keep the constituent packages but consolidate them all in one egg and one git repository. That is: we had ploneintranet.theme, ploneintranet.suite, ploneintranet.whatever and so on. These are still there but they now all live within ploneintranet. This has a toplevel configure.zcml which includes the suite (our policy package), which then includes all the rest.

This took some heroic git-filter-branch work by @jcbrand to merge all our separate git commit histories into a single repo, but the result was very much worth the effort.

So now we have only the one repository, which is a single egg, but that single egg contains ~21 separate python packages, each with their own genericsetup, their own configure.zcml etc. Having that as a single code base makes it hugely more easy to refactor code and move stuff around, and make coordinated changes.

The big upside from this apart from easier code management, is that your import locations do not change at all.

6

I feel better now :wink: I wonder also what others do.

is not that easy, even as I said all our packages are example.* that's not true in fact.

what I'm going to do is more or less the same @datakurre proposed:

  • move code from the theme to the policy package, until I can get rid of it
  • move the buildout configurations to the policy package also
  • at some point in the future try the rename approach or just add a new package and start moving all over again

I spent a couple of days last week working on this and there are way to many moving parts; my previous approach did not resulted on a working site and, IMO, when you start looking at weird Zope configuration errors you know is time to give up.