OSE Site Backup

=Overview= Our website uses mbackup as a backup tool. mbackup uses duplicity as a back-end.

Introduction
mbackup performs module-oriented backups.

Example: mbkp.sh forum-db mbkp.sh forum-root mbkp.sh forum-extra

The commands above will backup 3 different modules: one for the forum database, one for the forum root files (which don't change frequently) and one for extra forum files (uploaded files)

You define what each module should back up using these module configuration files:
 * module-name.pre - executes before the backup starts
 * module-name.post - executes after the backup ends
 * module-name.files - defines which files should be included and excluded, using the same syntax as duplicity
 * module-name.priv - contains sensitive information, like database passwords

Pre-Requisites
OSE's site is located on a shared server. We had to install duplicity and its libraries on a separate folder (hence identified by $D_ROOT). Before running mbackup, we need to prepare some environment variables by running $D_ROOT/init.sh:

% cat $D_ROOT/init.sh

BASEDIR="$(dirname $0)" export LD_LIBRARY_PATH=$D_ROOT/lib:$D_ROOT/usr/lib export PYTHONPATH=$D_ROOT/share/pyshared:$LD_LIBRARY_PATH export PATH=~/bin:$D_ROOT/bin:$PATH export HGRCPATH="$HOME/.hgrc"

Files and Folders
mbackup itself is installed on a folder identified as $MBKP_HOME.

On ~/.config/mbackup/ we have configuration files:

-rw-r--r-- 1 a_user a_group 1 2011-06-05 15:43 mbackup-config.sh drwxr-xr-x 2 a_user a_group 4 2011-06-05 16:03 modules/ drwx-- 2 a_user a_group 4 2011-06-05 16:03 priv/

On ~/.config/mbackup/modules we have the configuration for each module to be backed up: blog-db.pre     drupal-db.post         forum-db.pre       wiki-db.post blog-db-var.pre drupal-db.pre          home.files         wiki-db.pre civicrm-db.pre  drupal-sites.excludes  limesurvey-db.pre  wiki-root.excludes

On ~/.config/mbackup/priv we have sensitive configuration data (like db passwords): altroot.priv    drupal-db.priv     home.priv             wiki-extra-1.priv blog-cache.priv drupal-root.priv   limesurvey-db.priv    wiki-extra-2.priv blog-db.priv    drupal-sites.priv  limesurvey-root.priv  wiki-root.priv blog-extra.priv forum-db.priv      _mbackup.priv blog-root.priv  forum-extra.priv   site-root.priv civicrm-db.priv forum-root.priv    wiki-db.priv

mbackup-config.sh is the main configuration file:

cat ~/.config/mbackup/mbackup-config.sh

FORUM_ROOT="path/to/forum/root" BLOG_ROOT="path/to/blog/root" OSE_ROOT="path/to/main/site" MBKP_BASE_TARGET="file://$MBKP_LOCAL_DATA"
 * 1) !/bin/sh
 * 1) MBKP_BASE_TARGET="sftp://user@backup.domain.com/mbackup-modules"

_mbackup.priv contains other configurations:

cat ~/.config/mbackup/priv/_mbackup.priv set +x PASSPHRASE='the-passphrase' FTP_PASSWORD="the-ftp-password"
 * 1) Please choose the default passwords you want to use
 * 2) for protecting your backups.
 * 1) Default Gpg passphrase for Duplicity
 * 1) Default password for ssh / sftp
 * 1) Delete the line below in order to use mbackup:
 * 2) MBKP_FIRST_RUN=1

~/.local/share/mbackup/data/ contains a folder for each module, storing duplicity archives. Said folder is rsynce'd to a remote server.

~/.local/share/mbackup/cache/modules contains cache files used during backup (like database dumps)

Performing a Backup
We have some cron jobs that perform our backups. See the daily backup job below:

. $D_ROOT/init.sh # ~/bin/mbkp.sh limesurvey-db ~/bin/mbkp.sh limesurvey-root # ~/bin/mbkp.sh civicrm-db # ~/bin/mbkp.sh drupal-db ~/bin/mbkp.sh drupal-root ~/bin/mbkp.sh drupal-sites # ~/bin/mbkp.sh forum-db ~/bin/mbkp.sh forum-root ~/bin/mbkp.sh forum-extra # ~/bin/mbkp.sh blog-db ~/bin/mbkp.sh blog-root ~/bin/mbkp.sh blog-extra # ~/bin/mbkp.sh wiki-db ~/bin/mbkp.sh wiki-root ~/bin/mbkp.sh wiki-extra-1 ~/bin/mbkp.sh wiki-extra-2 # ~/bin/sync2bkpaccount.sh
 * 1) Sync to backup account