26 November 2007

Sh** happens - therefore everybody needs a backup strategy

Did you ever lost data due to an accidental "rm -r *" command? Or because your spouse just installad the newest service pack of any operating system which caused your machine to cease working?

I know about failures of humans and machines. I never want to loose digital assets due to such failures, therefore I designed and implemented a broad backup strategy.

But first things first. Let's start with the kinds of sh** that can happen:

Risks to Digital Assets

  • Theft: Some nice guy steals your precious notebook with your even more precious digital assets on its drive.

  • Damage: Your drive gets sick due to headcrash or an overdose of spilled coffee (I mostly stick to green tea, but they say that is equally fatal for harddrives).

  • Accidental deletion: Wether it is "rm -r *" on unix machines or its equivalent on graphical user interfaces - users sometimes (involuntarily) erase their own data. All right, you might get it back from the recycle bin... unless somebody cleans that one

  • Viruses or other data corruption: My special friends are office applications, killing large documents. I'm sure you've experienced that one yourself...

My Goals Concerning Backups

  • Continue working as fast as possible.

  • Highly automated backup - the fewer manual processes the better.

  • Cover different computers with various kinds of valueable data

  • Affordable. Keep the price-tag in personal range.

My Infrastructure

I'm working as freelancing IT-consultant in Germany, therefore I don't have too many machines to care for. The following diagram shows my infrastructure - which is primarily used by my wife (Cheffe Uli, for text processing, billing, accounting and other organizational tasks required to run a business).

  • My primary machine is a MacBook Pro

  • I use two different external USB drives. They contain all our digital music (approx 40GByte) and a copy of all our photos.

  • An old PC runs OpenSUSE Linux 10.2 (very smoothly!) and contains two 160 GByte drives, which I configured as RAID-1 (mirrored) with the SUSE Yast configuration utility. The OS itself is located on a third drive. A few directories of the RAID can be mounted via Samba on our other machines.

The Pillars of My Backup Strategy


  1. Of course I use a version control system (Subversion). I setup my (home) repository on our win-XP machine. Every project file I work on is regularly commited to subversion.

  2. I fall in love with CrashPlan, a Java-based (commercial) lifesaver to backup arbitrary files and directories to arbitrary other machines (which must have a free version of CrashPlan installed). I backup the following things:

    • My Mac's valueable data to our home PC and to my RAID-1 linux server.

    • My wife's data from the home-PC to my Mac and to the RAID-1 server.

    • Not on the diagram: I backup all our data to my parents PC.

  3. Every week I copy my complete MacBook-drive to one partition of an external drive (with SuperDuper). Uneven weeks are copied to partition #1, even weeks go to partition #2.

  4. All my valuable data (especially project-related files and photos) are copied to Amazon S3 (TM) offsite storage with a small utility called JungleDisk (a little webdav-server). JungleDisk encrypts the files, so nobody without my key can read my files.

  5. I copy the whole disk of our home-PC with Acronis TrueImage.

  6. (not on the diagram): I keep a USB-stick with a pre-configured JungleDisk installation with me at all times. In case I need to access some files, I just plugin the stick into ANY machine (Win, Mac or Linux) with an Internet connection - and can start working a few minutes later...

  7. (not on the diagram): I use Versomatic on the Mac (there is a Windows equivalent called RealTime Backup), which stores versions of all my working files in its own repository. Whenever I save a file in any application, Versomatic creates a new version in its repo... so I can go back in time (like TimeMachine, but I can configure Versomatic to just backup what I need!).

  8. (not on the diagram): I backup files with iBackup - creating timestamped backup-sets on external drives or sometimes even on CDs or DVDs. I tried several others, including DobryBackup, but iBackup is simply better...

  9. (not on the diagram): I keep a copy of SpinRite, the awesome disk analysis and repair utility from fabulous Steve Gibson. It can recover even bad disk failures (but is way to slow for USB-connected drives...)

Now what?

Let us review the list of risks from the beginning:

  • Theft: If somebody steals my Mac, I'll have to get a new one. Bad enough. I'll be productive within minutes due to my SuperDuper backups.

  • Damage: Every drive in our infrastructure is backed up somewhere. It might take a while to replace the computer, but I can easily restore every file, from JungleDisk or CrashPlan or Subversion or another of my copies.

  • Accidental deletion: It'l be fatal if I deleted my subversion repository, but even that is safely copied by CrashPlan to several destinations (and not automatically deleted there!). Looks like I can survive a few of my own brain-blackouts...

  • Viruses or other data corruption: See above. Several possibilities to get back to consistent versions, depending on the kind of damage.

I'm not fully automated in my backups - but that's ok for me.

Apart from those (technical) risks, what else can happen?

  • You forget your passwords? Print out a list of important passwords, seal it in an envelope and deposit with trusted friends.

  • Your house burns down. All computers within are destroyed. As long as you remember your JungleDisk password or your CrashPlan-ID, you're done... External backups have their merits...

  • I keep a number of boot-CD's ready... for example a grand-universal-windows-boot-disk


I'm pretty sure you guys come up with some risks I forgot. The major pillars in my backup strategy are CrashPlan, JungleDisk and redundancy...