Write programs that do one thing and do it well.

So I finally tired of implementing file rotation and retention functionality for backup scripts on an ad-hoc basis, and have sorted it out forever*.

To that end, here’s a Python script and class that handles rotating and retaining copies of files, with hourly, daily, weekly and monthly granularity.

Usage:

usage: rotate.py [-h] [-v] [-V] [-H HOURS] [-d DAYS] [-w WEEKS] [-m MONTHS]
                 [-n] [-l] [-r]
                 [directory] [file [file ...]]

positional arguments:
  directory             destination directory
  file                  files to rotate

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         be more verbose
  -V, --version         display version information and exit
  -H HOURS, --hours HOURS
                        number of hourly versions to keep
  -d DAYS, --days DAYS  number of daily versions to keep
  -w WEEKS, --weeks WEEKS
                        number of weekly versions to keep
  -m MONTHS, --months MONTHS
                        number of monthly versions to keep
  -n, --dry-run         dry run - do not make any changes
  -l, --hard-link       hard link duplicate files to save space
  -r, --remove          remove original files

rotate.py will attempt to read configuration directives from the following
files, with each subsequent file overriding directives, and command-line flags
overriding them again: `/etc/rotate`, `~/.rotate`, `./.rotate`

And as an example, here’s how I invoke it on my daily MySQL database dumps:

rotate.py --days 7 --weeks 4 --months 12 --hard-link --remove /home/backup/mysql /home/backup/mysql/dumps/*.sql.gz

A separate script creates a gzipped SQL dump for each database, and stores them at /home/backup/mysql/dumps/${database}.sql.gz. rotate.py, as invoked above, will keep 7 daily, 4 weekly and 12 monthly copies of these extracts, stored in /home/backup/mysql/{daily,weekly,monthly}. Duplicate dumps (e.g. a new weekly and daily copy are needed on the same run) are hard-linked to save space, and the original files are removed.

And now I never need to write this code again. Ah, sweet laziness.

* May not actually be sorted out forever. YMMV.