There are two types of people in the world. Those who have had critical hard drive failure and those who are yet to. Backups are important. Imagine the pain of explaining to a child why there are not photos of them before they were five years old simply because you didn’t have a backup strategy in place. Or losing a client because you lost their files.
My current backup strategy balances convenience, security and duplication. It has both onsite and offsite components.
Securing access
My 13" MacBook Pro is secured using a long and secure password and an encrypted hard drive. All offsite backups are likewise secured. You can get the files but do nothing with them.
Time Machine and Carbon Copy Cloner
Time Machine runs hourly to an external USB drive connected to a Airport Extreme. This drive is not encrypted. I also clone once a day to an external USB drive connected directly to my MacBook using Carbon Copy Cloner (CCC). This provides a bootable option in case of internal hard drive failure. It’s not encrypted either because I’ve had problems with the encrypted drive being mounted and unmounted properly in Mac OS Lion. I tried SuperDuper! as an alternative to CCC but find it too hard to trust software which overstates the amount it has backed up. Reporting 330GB when I have 250GB of data is not comforting.
Sparsebundles
Mac OS allows you to create a special type of file called a sparsebundle. It is a single file on the hard drive which can be mounted to behave like an additional drive. An advantage of sparsebundles is the ability to encrypt them. To mount the drive and access its contents you need a password. All critically secure files can be placed in the sparsebundle.
After a month I decided this wasn’t the solution for me. There were two reasons.
- Time Machine is not guaranteed to back up a mounted sparsebundle with integrity. In other words, although the backup would appear to work it may not have. It’s not practical to unmount the sparsebundle for each hourly Time Machine backup.
- To restore a file from Time Machine that exists in the sparsebundle you need to restore the whole sparsebundle. 15GB for a 250mb file is a waste of time.
What to do? Not fully practical for day-to-day usage, but perfect for security.
Sparsebundles when needed
I have encrypted the main internal drive of my laptop and move all critcal files out of the sparsebundle. Now they get picked up by Time Machine and CCC as needed. Access to backups is quick.
Then, every day or so I run a shell script to:
- Mount an encrypted sparsebundle called Lockbox.
- Copy all necessary files to it i.e., those which are critical and I can’t afford to lose in any circumstances.
- Unmount the encrypted sparsebundle.
- Copy Lockbox to a 32GB USB drive. This is an encrypted file. You can’t do anything with it.
- Copy Lockbox to the PC in the house. The advantage here is it now gets incorporated into the PC’s backup as well.
Staging backups
To have all this run smoothly I’ve made some changes to the default storage on my system. The majority of my files are stored and organsied within PersonalBrain. Almost nothing, outside of application support files and a few temporary use files exists outside of my PersonaBrain database. With 11,000+ thoughts in it, you can only imagine how critical this is to my operations.
PersonalBrain is stored in the root of my home directory. That is ~/PersonalBrain.
The other application I use is Bento. It has likewise been moved to the root of my home directory. To do this I had to create a symbolic link (see Bento Data on Multiple Machines for instructions).
I have also created a directory called Transient Backups. It’s used for the temporary backup files created by applications. I need a backup but not forever. 1Password, Omnifocus and TextExpander have all had their Preferences modified to move their default backup locations to this directory. Hazel runs on the Omnifocus files to keep them no more than two weeks old.
Transient Backups also stores PersonalBrain BrainZips, Bento exports, Address Book exports and my mail archive as created by Mail Steward App (I copy to this database, keeping all mail in Mail.app and gmail).
With this configuration Time Machine pick up all application data, plus transient backups made from time to time.
To the vault
The Lockbox file is kept in ~/Vaults. This directory is excluded from Time Machine’s backup.
A shell script (included below and based on Encrypted Remote Backups with Sparse Bundles) uses rsync to copy PersonalBrain, Bento and the Transient Backup directory into the Lockbox. After the first copy, rsync only copies what is changed and this speeds up the process.
To the USB and network
The same script then copies the Lockbox sparsebundle to a USB stick and network location. Again rsync is used. The internal structure of a sparsebundle consists of many 8mb files. rsync copies only those which have changed.
The script
I have saved this script in ~/Documents. To run it I open a Terminal window and kick it off with the backup option.
#!/bin/sh # # sparse_backup.sh # # Encrypted remote backup using Mac OS X Sparsebundle Disk Images # bubbaATbubba.org # VOLUMENAME="Lockbox" BASEPATH="/Users/DCB/"; IMAGEFILE="${BASEPATH}/Vaults/Lockbox.sparsebundle"; LOCALHOST="Lancelot" REMOTEHOST="Guinevere" REMOTEUSER="David Buchan" REMOTEDIR="/home/backupuser/backup/remote" LOGFILE="/tmp/backup.log" EXCLUDEFILE="/Users/DCB/rsync.excludes" USB="DeathStar" RET=0; # local rsync commands to sync local mac files w/ mounted sparsebundle # backup_local() { DATE=`date` echo "Local rsync starting: ${DATE}" 2>&1 >> ${LOGFILE} echo "Local rsync starting: ${DATE}" 2>&1 #### CHANGE EVERYTHING BELOW HERE #### echo "..PersonalBrain" 2>&1 rsync -avE --delete ${BASEPATH}/PersonalBrain /Volumes/Lockbox/ 2>&1 >> ${LOGFILE} echo "..Bento" 2>&1 rsync -avE --delete ${BASEPATH}/Bento /Volumes/Lockbox/ 2>&1 >> ${LOGFILE} echo "..Transient Backups" 2>&1 rsync -avE --delete ${BASEPATH}/Transient\ Backups /Volumes/Lockbox/ 2>&1 >> ${LOGFILE} ##mkdir -p /Volumes/${VOLUMENAME}/${HOSTNAME} 2>&1 >> ${LOGFILE} # v to q to make quiet ##rsync -var --exclude-from=${EXCLUDEFILE} --delete /Users/user /Volumes/${VOLUMENAME}/${HOSTNAME} 2>&1 >> ${LOGFILE} #### STOP WITH YOUR CHANGES #### DATE=`date` echo "Local rsync ending: ${DATE}" 2>&1 >> ${LOGFILE} echo "Local rsync ending: ${DATE}" 2>&1 } # rsync the sparsebundle to usb backup_usb() { # Uncomment this if you're using the alternative method mentioned in the documentation. # return 1 if [ -d "/Volumes/${USB}" ]; then DATE=`date` echo "USB rsync starting: ${DATE}" 2>&1 >> ${LOGFILE} echo "USB rsync starting: ${DATE}" 2>&1 rsync -avE --delete ${BASEPATH}/Vaults/Lockbox.sparsebundle /Volumes/${USB}/ 2>&1 >> ${LOGFILE} #rsync -e ssh -var --delete ${IMAGEFILE} ${REMOTEUSER}@${REMOTEHOST}:"${REMOTEDIR}" 2>&1 >> ${LOGFILE} DATE=`date` echo "USB rsync ending: ${DATE}" 2>&1 >> ${LOGFILE} echo "USB rsync ending: ${DATE}" 2>&1 else echo "/Volumes/${USB} not found...skipping" RET-1; fi } # rsync the sparsebundle offsite backup_offsite() { # Uncomment this if you're using the alternative method mentioned in the documentation. # return 1 if [ -d "/Volumes/Vaults" ]; then DATE=`date` echo "Offsite rsync starting: ${DATE}" 2>&1 >> ${LOGFILE} echo "Offsite rsync starting: ${DATE}" 2>&1 rsync -avO --exclude '.DS_Store' --delete ~/Vaults/ /Volumes/Vaults/ 2>&1 >> ${LOGFILE} #rsync -e ssh -var --delete ${IMAGEFILE} ${REMOTEUSER}@${REMOTEHOST}:"${REMOTEDIR}" 2>&1 >> ${LOGFILE} DATE=`date` echo "Offsite rsync ending: ${DATE}" 2>&1 >> ${LOGFILE} echo "Offsite rsync ending: ${DATE}" 2>&1 else RET=1; fi } # function to umount sparseimage unmount_sparse() { echo "Attempting to dismount ${IMAGEFILE}" 2>&1 if [ ! -d "/Volumes/${VOLUMENAME}" ]; then #echo "Volume /Volumes/${VOLUMENAME} not mounted" RET=1; else sleep 10 /usr/bin/hdiutil unmount "/Volumes/${VOLUMENAME}" 2>&1 >> ${LOGFILE} if [ -d "/Volumes/${VOLUMENAME}" ]; then echo "Unable to unmount volume /Volumes/${VOLUMENAME}" RET=1; fi fi } # function to mount sparseimage mount_sparse() { echo "Attempting to mount ${IMAGEFILE}" 2>&1 if [ ! -d "/Volumes/${VOLUMENAME}" ]; then open -W ${IMAGEFILE} 2>&1 >> ${LOGFILE} if [ ! -d "/Volumes/${VOLUMENAME}" ]; then echo "Unable to attach ${IMAGEFILE} / unable to mount /Volumes/${VOLUMENAME}." RET=1; fi fi } getpid() { PID=`ps -axwww | grep rsync | grep ${IMAGEFILE} | grep -v grep | awk '{print $1}'` if [ ${PID}0 -ne 0 ]; then echo "$0 already running (PID ${PID}); Exiting." exit 1; fi } case "$1" in mount) getpid mount_sparse exit $RET; ;; unmount) unmount_sparse exit $RET; ;; backup) getpid mount_sparse if [ "$RET" = "0" ]; then # do backup stuff here backup_local unmount_sparse if [ "$RET" = "0" ]; then backup_usb backup_offsite else exit $RET; fi else exit $RET; fi ;; offsite) backup_offsite ;; *) echo "Usage: $0 {mount|unmount|backup|offsite}" exit 1; ;; esac
