Simple #backup using #cron #tar and #LTFS on a #LTO tape drive

We all know back-up is important and a must have in case of critical systems.

Making a backup on tape is one of the best industry standards that is still unmatched.

In the following I am presenting a simple way to make tape back-ups using tar and LTFS on LTO5 and up tapes.

Why LTFS , because it will be easier to access the individual back-up files in case you have several automatic back-up scripts saving data on the same tape.

The back-up I am setting up has several different sets of data:

1. WordPress installation directory, less then 100MB
2. WordPress database dump, less then 100MB
3. OwnCloud installation directory, less then 100MB
4. OwnCloud database dump, less then 100MB
5. OwnCloud data directory, around 900MB

1-4 will be done as a full back-up each day. 5 will be done as a full back-up on Monday morning ( 3.a.m) and incremental back-up for the other days.

LTFS on LTO5 allows a maximum of 1330MB on a tape so gives me quite a big space without having to switch tapes daily.

I can use one tape for a week with a minimum rotation of 2 tapes. Why minimum of 2 tapes ?, because you want to have at least one full back-up on hand. If you want to have a longer history for your files you can increase the size of the tape set to 4 or 5 so that you have at least one whole month of history.

The extra benefit is that I need to change the tape only on Sunday so my trips to my data-center are limited.

STEP 1: Prepare the tape (to be done each Sunday afternoon)

### Every Sunday
# Unmount the tape if needed
umount /media/lto5
# Load a new tape
# Format tape as ltfs, this will erase all data
mkltfs -d /dev/st0 --force
ltfs -o devname=/dev/st0 /media/lto5

After executing the above steps you new LTFS tape is loaded under the /media/lto5 directory.

STEP 2: Setup the cron job for back-ups 1-4 (to be done once)

Add the following script under /etc/cron.daily

#!/bin/bash

# Path to binaries
TAR=/bin/tar

TAR_ARGS="--totals --blocking-factor=1024"
NOWYYYYMMDD=$(date -I)
OWNCLOUDTARNAME=$NOWYYYYMMDD.owncloud.tar
WORDPRESSTARNAME=$NOWYYYYMMDD.wordpress.tar

# owncloud directory backup
$TAR -czvf "/media/storage/backup/owncloud/$(date '+%F')-owncloud.tgz" /media/storage/www/html/owncloud/owncloud

# owncloud database backup
mysqldump -u homeclouddbuser -pxxxxx homeclouddb | gzip > "/media/storage/backup/owncloud/$(date '+%F')-ownclouddb.sql.gz"

# wordpress directory backup
$TAR -czvf "/media/storage/backup/wordpress/$(date '+%F')-wordpress.tgz" /media/storage/www/html/owncloud/wordpress

# wordpress database backup
mysqldump -u wpuser -pxxxxx wordpress | gzip > "/media/storage/backup/wordpress/$(date '+%F')-wordpressdb.sql.gz"

if mountpoint -q /media/lto5; then
    $TAR $TAR_ARGS -cpf /media/lto5/$OWNCLOUDTARNAME "/media/storage/backup/owncloud/$(date '+%F')"*
    $TAR $TAR_ARGS -cpf /media/lto5/$WORDPRESSTARNAME "/media/storage/backup/wordpress/$(date '+%F')"*.....
else
    echo "Tape not loaded skipping copy to tape"
fi

This is a very simple script that will perform 1-4 back-ups daily and dump the contents as tar files on the LTFS tape.

Note:

• we make first a dump of the data on another back-up location, on another phzsical disk “/media/storage/backup” for both wordpress and owncloud. This is to ensure we have a handy back-up for this critical elements on the server itself
• In a second step the already dumped data is tar-ed to the LTFS tape
• TAR_ARGS=”–totals –blocking-factor=1024″ that ensure that a proper speed of writing to the LTO tape is maintained and avoid shoe-shining. The blocking factor of 1024 is best for the LTO5 tapes . For higher LTO versions higher values may be more appropriate.

STEP 3: Setup the cron job for back-ups 5 (to be done once)

#!/bin/bash

# Log base directory
LOGBASE=/root/log

# Backup dirs; do not prefix /
BACKUP_ROOT_DIR="data"
BACKUP_PATH="/media/storage/www/"

# Get todays day like Mon, Tue and so on
NOW=$(date +"%a")
NOWYYYYMMDD=$(date -I)

# tar file names template
TARNAMEFULL=$NOWYYYYMMDD.full.tar
TARNAMEPART=$NOWYYYYMMDD.part.tar

# Exclude file and tar args
TAR_ARGS="--totals --blocking-factor=1024"
EXCLUDE_CONF=/root/.backup.exclude.conf

# Backup Log file
LOGFIILE=$LOGBASE/$NOW.backup.log

# Path to binaries
TAR=/bin/tar

# ------------------------------------------------------------------------
# Excluding files when using tar
# Create a file called $EXCLUDE_CONF using a text editor
# Add files matching patterns such as follows (regex allowed):
# home/gvoina/iso
# home/gvoina/*.cpp~
# ------------------------------------------------------------------------
[ -f $EXCLUDE_CONF ] && TAR_ARGS="-X $EXCLUDE_CONF"

#### Custom functions #####
# Make a full backup
full_backup(){
if mountpoint -q /media/lto5; then
    local old=$(pwd)
    cd $BACKUP_PATH
    echo $old
    echo "Writing file:"
    echo $TARNAMEFULL
    $TAR $TAR_ARGS -cpf /media/lto5/$TARNAMEFULL $BACKUP_ROOT_DIR
    cd $old
    else
        echo "Tape not available skipping backup"
fi
}

# Make a  partial backup
partial_backup(){
if mountpoint -q /media/lto5; then
    local old=$(pwd)
    cd $BACKUP_PATH
    echo "Writing file:"
    echo $TARNAMEPART
    # Copy only new data
    $TAR $TAR_ARGS -cpf /media/lto5/$TARNAMEPART -N "$(date -d '1 day ago')" $BACKUP_ROOT_DIR
    cd $old
    else
        echo "Tape not available skipping backup"
fi
}

#### Main logic ####

# Make sure log dir exits
[ ! -d $LOGBASE ] && $MKDIR -p $LOGBASE

# Okay let us start backup procedure
# If it is Monday make a full backup;
# For Mue to Sun make a partial backup
# Weekend no backups
case $NOW in
    Mon)        full_backup;;
    Tue|Wed|Thu|Fri|Sat|Sun)    partial_backup;;
    *) ;;
esac > $LOGFIILE 2>&1

This is a simple script that will guess the day of the week and then proceeds to create a full or incremental back-up.

Note:

• TAR_ARGS=”–totals –blocking-factor=1024″ that ensure that a proper speed of writing to the LTO tape is maintained and avoid shoe-shining. The blocking factor of 1024 is best for the LTO5 tapes . For higher LTO versions higher values may be more appropriate.
• Partial back-up is done by using a simple tar filter -N “$(date -d ‘1 day ago’)” that will take into account only the files changed in the last day. You can implement more complicated methods but this is a simple straight forward good enough method.

So remember, backing up on tape is still the gold standard of back-ups.