Backup without buildout

1

In Plone 5 we used collective.recipe.backup and it worked great. But it is buildout based and for Plone 6 we were planning on eschewing buildout. Is there an alternative option? The docs are blank Backup and Restore – Deployment — Plone Documentation v6.0 on this topic.

I think all collective.recipe.backup does is create scripts in the bin directory. I haven't tried this yet but I am guessing I could probably just create my own script adjusted to any config changes, and without the buildout sys.path part.

1 Like
2

In all our projects, we backup Data.fs + blobstorage completely every day...not entirely smart, but totally efficient and fool-proof...works for us with about 20 GB of data.

3

we use rsync and backup Data.fs and the blobstorage every night.

4

I know collective.recipe.backup uses rsync but I haven't looked into the details. I take it you mean you are running rsync commands directly without repozo?

5

collective.recipe.backup uses rsync with hard-links to backup the blobstorage. repozo is specific for Data.fs.

So, to strictly recreate what the recipe does one would have to use both repozo and rsync.

6

The best is to convert the recipe to a cookiecutter template, as done here for the zope recipe:

7

If you have an example you'd be willing to share, I'd love to see it. I think I have an ok understanding of how repozo works and what rsync does in the abstract, but keeping them in sync looks more complicated. Especially when considering full vs partial backups and how cleanup and rotation is done.

8

Yes, this would obviously be great but I do not have the expertise on what the old recipe is fully doing to be able to contribute much to making a template.

9

we use dirvish script to make backups, it makes a lot of things very easy. but a simple rsync command do the job also:

rsync -HAaxXP user@sourcemaschine:/opt/Plone/var/filestorage/ ./filestorage
rsync -HAaxXP user@sourcemaschine:/opt/Plone/var/blobstorage/ ./blobstorage
10

interesting, we were talking about this at the Midsumnmer sprint yesterday evening.

Note: whenever you do make rsync backups manually, always make sure to use the order as @1letter shows. Backup the Data.fs or filestorage first and then the blobstorage.

When you backup/rsync the blobstorage first, which will take the most time, new files/images could be uploaded to Plone, get a registration in the index and then have a blob created that your rsync 'misses'. So when you restore that backup you'll have missing blobs that the Data.fs references to.

In the correct order you might backup some extra blobs for which there is no reference yet in the Data.fs.Bbut at least all references to blobs in the Data.fs up until that point in time are accounted for in the backup.

11

with the dirvish script only initial happend a full backup, on every day later, it's only copy the "new diff" to the backup machine.
but yes, you are right. missing blobs can happend with this strategy.

12

A nicer solution to that ordering problem would be to take a snapshot of the file system first, then do the backup (yay ZFS). OK, not really practical unless you already are using ZFS.

I thought that it was safest to shut down the Zope before doing the backup.