Skip to content

· Operations

Backups are a promise you have not tested

A backup you have never restored is a hope, not a plan. Test it before the bad day decides for you.

I have watched two businesses learn the same lesson in the same week. Both had backups. One had tested them.

The first owner showed me a shelf of drives and a cloud dashboard full of green checkmarks. The backup software had reported success every night for two years. When the server died, we discovered the drives had been unreadable for months, because the software had been backing up to a folder on the same machine that failed. The green checkmarks were measuring effort, not results.

The second owner had a simpler setup and a weirder habit: once a quarter, he wiped a spare machine and restored everything onto it. It took him an afternoon. When his machine died, he was back at work the next morning, mildly annoyed instead of destroyed.

The difference was not money. It was one question: has anyone ever restored this backup and opened the files?

Backups fail two ways. The first is silent: a job stops running, a drive fills, a path changes, and every report keeps saying success. The second is at the moment of truth: the backup runs fine, but the restore was never practiced, so nobody knows the steps, the password, or the order. Both are fixed by the same boring habit: a scheduled restore drill.

The drill does not need to be heroic. Once a quarter, restore to a spare machine or a test folder. Open the files. Time it. Write down the steps, because the day you need them is not the day to learn them. Keep one copy offsite and one offline, because ransomware targets the backup too, and a backup on the same network is a hostage, not a rescue.

I built a small tool that automates the verification for GitHub-based backups, checking that a restore actually works and cleaning up after itself. It is open source at github.com/StefanYasin/restore-drill. The tool matters less than the habit it enforces.

If you cannot say “we restored from backup last month and it worked,” you do not have a backup plan. You have a promise. Promises are cheap until the invoice lands.

This is the first question we ask every Cloud and Cable client, before we touch a cable: show me a restore that worked.

Written by Stefan Yasin, Hamilton, Ontario.

← All writing