Most backup failures are discovered on the day you need the backup: the job stopped months ago, the password is lost, or the copy turns out to be incomplete. All three are cheap to catch in advance.
Three copies, in three places
- A drive at home or in the office, for fast restores.
- A second computer, so one broken machine or drive never takes everything with it.
- Rented storage in another country, for fire, flood and theft.
Every copy is encrypted on your own machine before it leaves, so the storage company holds data it can't read. We use restic, which is free and open source; it encrypts, removes duplicate data and keeps dated snapshots you can go back to. Keep the encryption password in your password manager and on paper: without it the backups are useless, to a thief and to you.
Make the off-site copy add-only
If a machine is compromised, an attacker who can reach your backups can delete them too. Set the off-site copy up so the machines that send backups can add new snapshots but cannot delete or overwrite old ones. Clearing out old snapshots then becomes a deliberate, manual step.
The restore drill
Once a month, restore something real and compare it with the original:
- Pick some files at random, restore them, and check they match byte for byte.
- For a database, restore it somewhere separate and count the rows. Our last records drill brought back 9,484 of 9,484 rows.
- For a whole system, such as a mail server, start a throwaway copy from the backup and check that it works. When we did this for our own mail server, the restored copy had every account and a test message we had planted beforehand.
Write down the steps the first time. The day you need them is not the day to work them out.
Let silence be the alarm
Each backup job should report when it succeeds, and a daily check should raise an alarm when a report is missing or old. Don't wait for an error message: a job that has stopped running sends none.