Write programs that do one thing and do it well.
So I finally tired of implementing file rotation and retention functionality for backup scripts on an ad-hoc basis, and have sorted it out forever*.
To that end, here’s a Python script and class that handles rotating and retaining copies of files, with hourly, daily, weekly and monthly granularity.
Usage:
usage: rotate.py [-h] [-v] [-V] [-H HOURS] [-d DAYS] [-w WEEKS] [-m MONTHS]
[-n] [-l] [-r]
[directory] [file [file ...]]
positional arguments:
directory destination directory
file files to rotate
optional arguments:
-h, --help show this help message and exit
-v, --verbose be more verbose
-V, --version display version information and exit
-H HOURS, --hours HOURS
number of hourly versions to keep
-d DAYS, --days DAYS number of daily versions to keep
-w WEEKS, --weeks WEEKS
number of weekly versions to keep
-m MONTHS, --months MONTHS
number of monthly versions to keep
-n, --dry-run dry run - do not make any changes
-l, --hard-link hard link duplicate files to save space
-r, --remove remove original files
rotate.py will attempt to read configuration directives from the following
files, with each subsequent file overriding directives, and command-line flags
overriding them again: `/etc/rotate`, `~/.rotate`, `./.rotate`
And as an example, here’s how I invoke it on my daily MySQL database dumps:
rotate.py --days 7 --weeks 4 --months 12 --hard-link --remove /home/backup/mysql /home/backup/mysql/dumps/*.sql.gz
A separate script creates a gzip
ped SQL dump for each database, and stores them at /home/backup/mysql/dumps/${database}.sql.gz
. rotate.py
, as invoked above, will keep 7 daily, 4 weekly and 12 monthly copies of these extracts, stored in /home/backup/mysql/{daily,weekly,monthly}
. Duplicate dumps (e.g. a new weekly and daily copy are needed on the same run) are hard-linked to save space, and the original files are removed.
And now I never need to write this code again. Ah, sweet laziness.
* May not actually be sorted out forever. YMMV.