Upgrade steps that need to survive server restarts

Hi all,

every now and then we need to do some upgrade steps that have to reindex lots of content (100k to 1Milion objects).

We have been using different approaches so far:

  • write the UUID of each reindexed object in a file and upon a server restart load the file again to check if it needs to be reindexed (downsides: you need to clear the file once it finished, make sure the file is not removed...)
  • add a marker interface on the object and discriminate for that interface on the catalog query (downside: you need to reindex all the objects again to remove the marker interface once the upgrade is done)

But today I thought about something different:

  • create an ASCIILine registry record that holds a 'START' value
  • the upgrade step checks the record value and keeps updating its value as long as its successfully reindexing objects, writing the value of how many objects have been reindexed so far
  • if it finishes, updates the value to 'DONE' so that at the very beginning of the upgrade step it can check if it's already in that state and do nothing

The upsides of this is that you can control and see the value on /plone_registry itself and specially it survives restarts happily without having to use marker interfaces or external files that need clean up afterwards.

Do you have other approaches? Any downsides?

check this thread:

This sounds like a typical queueing task (your file based approach in fact implements a queue). There are solutions to implement a queue in the ZODB (e.g. zc.queue, z3c.taskqueue). With an appropriate design (i.e. a general task executor), there would be no need to delete the queue at the end; it would just become empty.

If your Zope accepts requests before the reindexing is complete, new objects may be created. While they would not need reindexing, the reindexing process may not know that. As a consequence, you may not know the total number of reindexed objects and your must be careful with the termination check.