26 November 2007

Sh** happens - therefore everybody needs a backup strategy

Did you ever lost data due to an accidental "rm -r *" command? Or because your spouse just installad the newest service pack of any operating system which caused your machine to cease working?

I know about failures of humans and machines. I never want to loose digital assets due to such failures, therefore I designed and implemented a broad backup strategy.

But first things first. Let's start with the kinds of sh** that can happen:

Risks to Digital Assets

  • Theft: Some nice guy steals your precious notebook with your even more precious digital assets on its drive.

  • Damage: Your drive gets sick due to headcrash or an overdose of spilled coffee (I mostly stick to green tea, but they say that is equally fatal for harddrives).

  • Accidental deletion: Wether it is "rm -r *" on unix machines or its equivalent on graphical user interfaces - users sometimes (involuntarily) erase their own data. All right, you might get it back from the recycle bin... unless somebody cleans that one

  • Viruses or other data corruption: My special friends are office applications, killing large documents. I'm sure you've experienced that one yourself...

My Goals Concerning Backups

  • Continue working as fast as possible.

  • Highly automated backup - the fewer manual processes the better.

  • Cover different computers with various kinds of valueable data

  • Affordable. Keep the price-tag in personal range.

My Infrastructure

I'm working as freelancing IT-consultant in Germany, therefore I don't have too many machines to care for. The following diagram shows my infrastructure - which is primarily used by my wife (Cheffe Uli, for text processing, billing, accounting and other organizational tasks required to run a business).

  • My primary machine is a MacBook Pro

  • I use two different external USB drives. They contain all our digital music (approx 40GByte) and a copy of all our photos.

  • An old PC runs OpenSUSE Linux 10.2 (very smoothly!) and contains two 160 GByte drives, which I configured as RAID-1 (mirrored) with the SUSE Yast configuration utility. The OS itself is located on a third drive. A few directories of the RAID can be mounted via Samba on our other machines.

The Pillars of My Backup Strategy


  1. Of course I use a version control system (Subversion). I setup my (home) repository on our win-XP machine. Every project file I work on is regularly commited to subversion.

  2. I fall in love with CrashPlan, a Java-based (commercial) lifesaver to backup arbitrary files and directories to arbitrary other machines (which must have a free version of CrashPlan installed). I backup the following things:

    • My Mac's valueable data to our home PC and to my RAID-1 linux server.

    • My wife's data from the home-PC to my Mac and to the RAID-1 server.

    • Not on the diagram: I backup all our data to my parents PC.

  3. Every week I copy my complete MacBook-drive to one partition of an external drive (with SuperDuper). Uneven weeks are copied to partition #1, even weeks go to partition #2.

  4. All my valuable data (especially project-related files and photos) are copied to Amazon S3 (TM) offsite storage with a small utility called JungleDisk (a little webdav-server). JungleDisk encrypts the files, so nobody without my key can read my files.

  5. I copy the whole disk of our home-PC with Acronis TrueImage.

  6. (not on the diagram): I keep a USB-stick with a pre-configured JungleDisk installation with me at all times. In case I need to access some files, I just plugin the stick into ANY machine (Win, Mac or Linux) with an Internet connection - and can start working a few minutes later...

  7. (not on the diagram): I use Versomatic on the Mac (there is a Windows equivalent called RealTime Backup), which stores versions of all my working files in its own repository. Whenever I save a file in any application, Versomatic creates a new version in its repo... so I can go back in time (like TimeMachine, but I can configure Versomatic to just backup what I need!).

  8. (not on the diagram): I backup files with iBackup - creating timestamped backup-sets on external drives or sometimes even on CDs or DVDs. I tried several others, including DobryBackup, but iBackup is simply better...

  9. (not on the diagram): I keep a copy of SpinRite, the awesome disk analysis and repair utility from fabulous Steve Gibson. It can recover even bad disk failures (but is way to slow for USB-connected drives...)

Now what?

Let us review the list of risks from the beginning:

  • Theft: If somebody steals my Mac, I'll have to get a new one. Bad enough. I'll be productive within minutes due to my SuperDuper backups.

  • Damage: Every drive in our infrastructure is backed up somewhere. It might take a while to replace the computer, but I can easily restore every file, from JungleDisk or CrashPlan or Subversion or another of my copies.

  • Accidental deletion: It'l be fatal if I deleted my subversion repository, but even that is safely copied by CrashPlan to several destinations (and not automatically deleted there!). Looks like I can survive a few of my own brain-blackouts...

  • Viruses or other data corruption: See above. Several possibilities to get back to consistent versions, depending on the kind of damage.

I'm not fully automated in my backups - but that's ok for me.

Apart from those (technical) risks, what else can happen?

  • You forget your passwords? Print out a list of important passwords, seal it in an envelope and deposit with trusted friends.

  • Your house burns down. All computers within are destroyed. As long as you remember your JungleDisk password or your CrashPlan-ID, you're done... External backups have their merits...

  • I keep a number of boot-CD's ready... for example a grand-universal-windows-boot-disk


I'm pretty sure you guys come up with some risks I forgot. The major pillars in my backup strategy are CrashPlan, JungleDisk and redundancy...


dov said...

I wanted to let you know about a cool backup program that offers FREE and UNLIMITED backup.

With Zoogmo you control where your files are backed up by building a backup network of your own or your friends’ computers. You can even backup to an external hard drive of your choice.

Zoogmo runs silently in the background and automatically backs up ALL of your data over the internet, using a combination of Triple-DES and AES 256 encryption to make sure that only you have access to your files.

Check out Zoogmo at www.zoogmo.com to download Zoogmo for free and begin turning your social network into your backup network.

Remember, with Zoogmo you have no annoying monthly fees - totally free unlimited backup you can trust!

Dr. Gernot Starke said...

Zoogmo is Windows only.

DW said...

Nice post describing your backup strategy. I am a huge fan of CrashPlan also -- it is intrinsically superior to the many online backup services (mozy, etc.) as long as you use it to also backup to a remote machine you can access if needed.

I have WHS (windows home server) doing its nightly snapshots to get full image backups of all machines, then CrashPlan sending files from my machine and the WHS to a server at my office (and vice versa).

One question about your plan: what is the benefit of JungleDisk/S3 since you have CrashPlan to handle off-site backups? Just for extra redundancy (versus just adding another destination machine to CrashPlan)?

Dr. Gernot Starke said...

DW: regarding JungleDisk: I got a copy of JungleDisk including all configuration stuff on a portable USB stick...

So, even when I'm away from my notebook, I can access all important files via this USB-stick - which provides another grain of security.

Anonymous said...

I think the best solution to save my data is using on line back up and full system recovery data softweres, that use encryptions security system. At the moment i'm using a new softwre from Italy: Memopal (www.memopal.com), where i can store 250GB and access from anywhere.

Anonymous said...

Ola Gernot, bad enough to say - but the Mac Laptop might be the worst risk in your equation. If your laptop gets stolen restoring a new machine would be the least of my headaches. But as FileVault for the Mac is such a mess that apparently you do not even mention it (and besides - I tried it and discontinued using it due to frequent data loss and disk corruption) the worst risk might be, that somebody offers you to buy back your data. How about this one - Cheers, Wolfgang