OSE Site Backup
Contents
Page Status
Hint: As of 2017, all of this information appears to be invalid. See OSE Server#Backups for current backup info.
See OSE Server#Backups for current backup info.
Overview
Our website uses mbackup as a backup tool. mbackup uses duplicity as a back-end.
Introduction
mbackup performs module-oriented backups.
Example:
mbkp.sh forum-db mbkp.sh forum-root mbkp.sh forum-extra
The commands above will backup 3 different modules: one for the forum database, one for the forum root files (which don't change frequently) and one for extra forum files (uploaded files)
You define what each module should back up using these module configuration files:
- module-name.pre - executes before the backup starts
- module-name.post - executes after the backup ends
- module-name.files - defines which files should be included and excluded, using the same syntax as duplicity
- module-name.priv - contains sensitive information, like database passwords
Pre-Requisites
OSE's site is located on a shared server. We had to install duplicity and its libraries on a separate folder (hence identified by $D_ROOT). Before running mbackup, we need to prepare some environment variables by running $D_ROOT/init.sh:
% cat $D_ROOT/init.sh
BASEDIR="$(dirname $0)" export LD_LIBRARY_PATH=$D_ROOT/lib:$D_ROOT/usr/lib export PYTHONPATH=$D_ROOT/share/pyshared:$LD_LIBRARY_PATH export PATH=~/bin:$D_ROOT/bin:$PATH export HGRCPATH="$HOME/.hgrc"
Files and Folders
mbackup itself is installed on a folder identified as $MBKP_HOME.
On ~/.config/mbackup/ we have configuration files:
-rw-r--r-- 1 a_user a_group 1 2011-06-05 15:43 mbackup-config.sh drwxr-xr-x 2 a_user a_group 4 2011-06-05 16:03 modules/ drwx------ 2 a_user a_group 4 2011-06-05 16:03 priv/
On ~/.config/mbackup/modules we have the configuration for each module to be backed up:
blog-db.pre drupal-db.post forum-db.pre wiki-db.post blog-db-var.pre drupal-db.pre home.files wiki-db.pre civicrm-db.pre drupal-sites.excludes limesurvey-db.pre wiki-root.excludes
On ~/.config/mbackup/priv we have sensitive configuration data (like db passwords):
altroot.priv drupal-db.priv home.priv wiki-extra-1.priv blog-cache.priv drupal-root.priv limesurvey-db.priv wiki-extra-2.priv blog-db.priv drupal-sites.priv limesurvey-root.priv wiki-root.priv blog-extra.priv forum-db.priv _mbackup.priv blog-root.priv forum-extra.priv site-root.priv civicrm-db.priv forum-root.priv wiki-db.priv
mbackup-config.sh is the main configuration file:
cat ~/.config/mbackup/mbackup-config.sh
#!/bin/sh FORUM_ROOT="path/to/forum/root" BLOG_ROOT="path/to/blog/root" OSE_ROOT="path/to/main/site" # MBKP_BASE_TARGET="sftp://user@backup.domain.com/mbackup-modules" MBKP_BASE_TARGET="file://$MBKP_LOCAL_DATA"
_mbackup.priv contains other configurations:
cat ~/.config/mbackup/priv/_mbackup.priv # Please choose the default passwords you want to use # for protecting your backups. set +x # Default Gpg passphrase for Duplicity PASSPHRASE='the-passphrase' # Default password for ssh / sftp FTP_PASSWORD="the-ftp-password" # Delete the line below in order to use mbackup: # MBKP_FIRST_RUN=1
~/.local/share/mbackup/data/ contains a folder for each module, storing duplicity archives. Said folder is rsynce'd to a remote server.
~/.local/share/mbackup/cache/modules contains cache files used during backup (like database dumps)
Performing a Backup
We have some cron jobs that perform our backups. See the daily backup job below:
. $D_ROOT/init.sh # ~/bin/mbkp.sh limesurvey-db ~/bin/mbkp.sh limesurvey-root # ~/bin/mbkp.sh civicrm-db # ~/bin/mbkp.sh drupal-db ~/bin/mbkp.sh drupal-root ~/bin/mbkp.sh drupal-sites # ~/bin/mbkp.sh forum-db ~/bin/mbkp.sh forum-root ~/bin/mbkp.sh forum-extra # ~/bin/mbkp.sh blog-db ~/bin/mbkp.sh blog-root ~/bin/mbkp.sh blog-extra # ~/bin/mbkp.sh wiki-db ~/bin/mbkp.sh wiki-root ~/bin/mbkp.sh wiki-extra-1 ~/bin/mbkp.sh wiki-extra-2 # # Sync to backup account ~/bin/sync2bkpaccount.sh
