For many years I have been struggling with external storage space to daily backup my ZODB Data.fs projects until I found Borg Backup (https://www.borgbackup.org/). It is truly a incremental backup even for the huge Data.fs, really save me a lot of backup space.
Give it a try.
I guess collective.recipe.backupprovides incremental backup, by using the repozoutility.
Could you please elaborate more on the advantages of BorgBackup over collective.recipe.backup?
My use case is I need to backup Data.fs everyday for many years. The storage would be humongous even with collective.recipe.backup.
Borg can also backup blobs storage effectively.
Borg Backup can be a supplement or replacement of collective.recipe.backup.
I see, thank you for the information.
collective.recipe.backup also backup blobs inteligently, by using hardlinks (the underlying file system must support this feature).
borg backup does a backup of the data.fs in the filestorage but there are also sites using relstorage.
I leave out snapshots of the server, but they also need stopping the services and finish transactions before taking a snapshot. Your goal is deduplication and lower storage footprint and faster execution.
When you backup a database without proper freezing the state during backup (e.g., using a LVM Snapshot) you can backup, but its maybe corrupted). You need at least shut down operations and need to avoid a running transaction and stop new ones until the LVM snapshot is frozen.
collective.recipe.backup and the repozo command take care of this.
There is no replacement yet for dockerized setups using relstorage and e.g. PostgreSQL.
You can use the backup commands for youb database system. But the challenge to avoid inconsistent states remains.
Did you ever test successful restoring the backups taken this way?
You do not need just backup, you need restore one day!
For Data.fs + Blobstorage collective.recipe.backup dies an amazing job for exactly your purpose, wenn you use the incremental options. If you cannot benefit from drastically deduplication, maybe your setup is matching the level of understanding the risks of using plain file backups of database files, even hen they are incremental. Borg backup is an excellent tool and has nice deduplication features. but even that can shoot you into the foot if you have not multiple targets. If one block in the backup is broken, all symlinked instances are too.
For RelStroage with Postgresql I can recommend Barman User Guide — Barman 3.20.1 documentation