March 1, 2020 · backup software

Using BorgBackup Efficiently

This is another quick-and-dirty article on how to use BorgBackup software. Borg is a tool that abstracts away the details of snapshotting, encrypting, and deduplicating your backup. That last feature, deduplicating, is especially important since you don't want to backup the same data more than once. I'm using Borg to backup my MacBook, as well as backup my Keybase File System.

Start by installing Borg on the computer you wish to backup files from, as well as the computer containing the backup storage device. Although we'll be using an external server, Borg has full functionality on a single computer. I'm using rsync.net since they have Borg support, and a cheaper Borg only plan at only $0.015/GB/month.

Borg has deduplication and compression enabled by default, so we won't worry about configuring them. The encryption standard used by Borg is AES-256, which doesn't benefit from having a key/passphrase with more than 256 bits of entropy, so we will be creating a 256 bit key. Also, of the two key storage choices, we'll be using repokey, which is just as secure as keyfile since we are using a strong passphrase.

First we create the passphrase on our local machine, and set it to be readable only by the current user. Then we print the passphrase to the screen.

# 32 bytes -> 256 bits
head -c 32 /dev/urandom | base64 > ~/.borg-passphrase
chmod 400 ~/.borg-passphrase
cat ~/.borg-passphrase

Now we can initialize the Borg repo on the remote server, at our specified path. Here, I initialize it at kbfs_backup which is expanded to ~/kbfs_backup. If you are using rsync.net, make sure to first run the command export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1 for every new shell that you use.

# if using rsync.net, export BORG_REMOTE_PATH=...
# borg init ... <user>@<server>:<folder-path>
borg init --encryption=repokey-blake2 3205@usw-s003.rsync.net:kbfs_backup

When prompted, paste in the passphrase from earlier. Now, we can create our backup script. Start by copying the script below into a new file. If you aren't using rsync.net, delete the line containing BORG_REMOTE_PATH. Edit the BORG_REPO and borg create lines to match your configuration. Here, Bash substitution is used to dynamically create the <snapshot-name>. Edit <backup-source> to be the path of the directory you want to backup.

#!/usr/bin/env bash
export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
export BORG_PASSCOMMAND="cat $HOME/.borg-passphrase"
export BORG_REPO='3205@usw-s003.rsync.net:kbfs_backup'

# borg create ... ::<snapshot-name> <backup-source>
borg create --progress --files-cache=ctime,size --noctime --noflags ::kbfs-$(date +%s) /keybase/private/yoonsikp/

borg prune --keep-daily 7 --keep-weekly 8 --keep-monthly 12 --keep-yearly 15

Save the script, and feel free to add it to your crontab so that it runs regularly.

Here's an explanation of each line.

  • BORG_REMOTE_PATH is the location of the Borg executable on the remote server. This is required because rsync.net has a different path for the newest Borg.

  • BORG_PASSCOMMAND loads the passphrase from your home directory, so that it doesn't need to be typed in.

  • BORG_REPO sets the default repository for Borg, i.e. the destination server and repo name, so that in later commands they can both be omitted. Later in borg create, we can use ::<snapshot-name>, as opposed to <repo-path>::<snapshot-name>.

  • For borg create, the --files-cache=ctime,size flag is used since I'm using a networked file-system which doesn't have stable inodes. If your source file-system is physical (hard-drive), you can remove this flag. We also use the --noatime, --noctime, and --nobsdflags, since these attributes are rarely useful.

  • Lastly, the borg prune command deletes old backups using a schedule, for example, keeping only a single backup for each year for backups older than 12 months, up to 15 years.

For further reading, check out the documentation on borg serve to enable features such as append-only and forced commands.

Update Nov 2022: removed deprecated borg flags