Re: Linux Backup with Modification Date Filter?

For completeness, I am following up on my original question regarding
the need for a Linux based backup solution that uses modification dates
as the basis for an exclude filter.

I was able to use rsnapshot, which uses rsync, and took advantage of
its exclude_file filter feature.

I wrote a bash script that acts as a wrapper to rsnapshot which first
generates a modification-date filter, then applies rsnapshot to the
resulting file list.

The script is called rsnapshot_backup.sh and sources (includes)
bash_routines.incl. Both files are listed below. I've also included a
sample email generated on successfully running this script.

rich

_____________________________________________________________________________

RSNAPSHOT_BACKUP
rsnapshot backup with modification-date filtering
release 0.90

copyright (C) 2006 business learning incorporated
_____________________________________________________________________________

script start: 2006-06-26 23:04:01

checking samba folder for backup...
samba folder (/mnt/shares/username) confirmed

mounting samba share...
share (//main/username) mounted

creating exclude file...
exclude file (/home/username/rsnapshot_username.excl) created

running rsnapshot (rsnapshot -c /home/username/rsnapshot_username.conf
daily)...
rsnapshot successful

unmounting samba share...
share (//main/username) unmounted

script end: 2006-06-26 23:04:08
_____________________________________________________________________________

script execution time: 00:00:07


RSNAPSHOT_BACKUP.SH:

#!/bin/bash

##############################################################################
#
# RSNAPSHOT_BACKUP
# rsnapshot backup with modification-date filtering
# release 0.90
#
# copyright (C) 2006 business learning incorporated
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
#
##############################################################################
#
# DESCRIPTION:
#
# archive files over samba with modification date earlier than
# BACKUP_START_DATE using rsnapshot and the rsync exclude_file list
# (EXCLUDE_FILE), and optionally (EMAIL_SEND) emails script
diagnostics
# to user (EMAIL_ADDR) upon script completion
#
# USAGE:
#
# rsnapshot_backup rsnapshot_backup_point rsnapshot_backup_interval
# e.g.: rsnapshot_backup my_backup_point daily
#
# where: rsnapshot_backup_point: backup locations per 'man rsnapshot'
# rsnapshot_backup_interval: backup interval per 'man
rsnapshot'
#
# REQUIRES:
#
# - rsnapshot package (tested against release 1.2.9), and
dependencies
# - rsnapshot.conf (RSNAPSHOT_CONF) with the following values set:
#
# exclude_file (enabled): EXCLUDE_FILE as generated in this
script
# e.g.: /home/username/rsnapshot_example.excl
#
# all other rsnapshot.conf values must be configured accordingly
before
# script use, per 'man rsnapshot'
#
# - user edits to script are made only in the section labeled "user
defines"
# - script must be run as root (for share access privileges)
# - bash_routines.incl must be co-located with this script
#
# NOTES:
#
# for rsnapshot to run its own diagnostics against RSNAPSHOT_CONF
# (e.g., rsnapshot -c rsnapshot_example.conf du), rsnapshot expects
# BACKUP_MOUNT_POINT and EXCLUDE_FILE to be persistent
#
##############################################################################

#
# includes
#
.. bash_routines.incl

#
# user defines
#
SNAPSHOT_NAME_ARGS=(diane rich)
SNAPSHOT_INTERVAL_ARGS=(daily weekly monthly)

check_for_root_user
check_arg_syntax "$1" "$2"

BACKUP_START_DATE="0601220000" # 2006-01-22
SCRIPT_PATH="/home/username/"
RSNAPSHOT_PATH="/usr/local/bin/"
EMAIL_SEND=1
EMAIL_ADDR="username@xxxxxxxxx"

case "$1" in
${SNAPSHOT_NAME_ARGS[0]})
BACKUP_MOUNT_POINT="/mnt/shares/Diane"
SAMBA_SHARE="//main/diane"
SAMBA_USERNAME="samba_username"
SAMBA_PASSWORD=""
;;
${SNAPSHOT_NAME_ARGS[1]})
BACKUP_MOUNT_POINT="/mnt/shares/Rich"
SAMBA_SHARE="//main/rich"
SAMBA_USERNAME="samba_username"
SAMBA_PASSWORD=""
;;
"*")
exit 1
;;
esac

#
# script defines
#
RSNAPSHOT_CONF="${SCRIPT_PATH}rsnapshot_${1}.conf"
EXCLUDE_FILE="${SCRIPT_PATH}rsnapshot_${1}.excl"
LOG_FILE="${SCRIPT_PATH}rsnapshot_${1}.log"
BACKUP_START_DATE_REF_FILE="${SCRIPT_PATH}rsnapshot_${1}.ref"
EMAIL_SUBJ_SUCCESS="Backup of $SAMBA_SHARE $2"
EMAIL_SUBJ_FAILURE="Backup of $SAMBA_SHARE $2 FAILED"
SCRIPT_FAIL=0
SCRIPT_SAMBAMOUNT_FAIL=0
SCRIPT_START_TIME=$(date +%s)
SCRIPT_END_TIME=$SCRIPT_START_TIME

#
# create/open log file
#
echo
"_____________________________________________________________________________"
| tee $LOG_FILE
echo "" | tee -a $LOG_FILE
echo "RSNAPSHOT_BACKUP" | tee -a $LOG_FILE
echo "rsnapshot backup with modification-date filtering" | tee -a
$LOG_FILE
echo "release 0.90" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
echo "copyright (C) 2006 business learning incorporated" | tee -a
$LOG_FILE
echo
"_____________________________________________________________________________"
| tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
echo "script start: `date +%Y-%m-%d" "%T`" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE

#
# create/confirm samba folder
#
echo "checking samba folder for backup..." | tee -a $LOG_FILE
if [ ! -d $BACKUP_MOUNT_POINT ]; then
mkdir $BACKUP_MOUNT_POINT

if [ $? -eq 0 ]; then
echo "no samba folder found, folder ($BACKUP_MOUNT_POINT) created"
| tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
else
echo "samba folder ($BACKUP_MOUNT_POINT) creation FAILED" | tee -a
$LOG_FILE
echo "" | tee -a $LOG_FILE
SCRIPT_FAIL=1
fi

else
echo "samba folder ($BACKUP_MOUNT_POINT) confirmed" | tee -a
$LOG_FILE
echo "" | tee -a $LOG_FILE
fi

#
# mount samba share
#
if [ $SCRIPT_FAIL -eq 0 ]; then
echo "mounting samba share..." | tee -a $LOG_FILE
smbmount $SAMBA_SHARE $BACKUP_MOUNT_POINT -o
username=$SAMBA_USERNAME,password=$SAMBA_PASSWORD

if [ $? -eq 0 ]; then
echo "share ($SAMBA_SHARE) mounted" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
else
echo "share ($SAMBA_SHARE) mount FAILED" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
SCRIPT_FAIL=1
SCRIPT_SAMBAMOUNT_FAIL=1
fi

fi

#
# create exclude filter list (based on BACKUP_START_DATE_REF_FILE)
#
if [ $SCRIPT_FAIL -eq 0 ]; then
echo "creating exclude file..." | tee -a $LOG_FILE

touch -t "$BACKUP_START_DATE" "$BACKUP_START_DATE_REF_FILE"
find "$BACKUP_MOUNT_POINT" -not -newer "$BACKUP_START_DATE_REF_FILE"
files_to_exclude
sed -e "s|^$BACKUP_MOUNT_POINT|- $BACKUP_MOUNT_POINT|"
files_to_exclude > "$EXCLUDE_FILE"
rm -f "$BACKUP_START_DATE_REF_FILE"
rm -f files_to_exclude
echo "exclude file ($EXCLUDE_FILE) created" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
fi

#
# run rsnapshot backup
#
if [ $SCRIPT_FAIL -eq 0 ]; then
echo "running rsnapshot (rsnapshot -c $RSNAPSHOT_CONF $2)..." | tee
-a $LOG_FILE
"$RSNAPSHOT_PATH"rsnapshot -c $RSNAPSHOT_CONF $2

if [ $? -eq 0 ]; then
echo "rsnapshot successful" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
else
echo "rsnapshot FAILED" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
SCRIPT_FAIL=1
fi

fi

#
# unmount samba share
#
if [ $SCRIPT_SAMBAMOUNT_FAIL -eq 0 ]; then
echo "unmounting samba share..." | tee -a $LOG_FILE
smbumount $BACKUP_MOUNT_POINT

if [ $? -eq 0 ]; then
echo "share ($SAMBA_SHARE) unmounted" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
else
echo "share ($SAMBA_SHARE) unmount FAILED" | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
SCRIPT_FAIL=1
fi

fi

#
# calculate script execution time
#
SCRIPT_END_TIME=$(date +%s)
END_SCRIPT="script end: `date +%Y-%m-%d" "%T`"
SCRIPT_DURATION=$[$SCRIPT_END_TIME - $SCRIPT_START_TIME]
HOURS=$[$SCRIPT_DURATION / 3600]
MINUTES=$[($SCRIPT_DURATION / 60) - ($HOURS * 60)]
SECONDS=$[$SCRIPT_DURATION % 60]

#
# close log file
#
echo $END_SCRIPT | tee -a $LOG_FILE
echo
"_____________________________________________________________________________"
| tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE
printf "script execution time: %02d:%02d:%02d\n" $HOURS $MINUTES
$SECONDS | tee -a $LOG_FILE
echo "" | tee -a $LOG_FILE

#
# email results
#
if ([ $SCRIPT_FAIL -eq 0 ] && [ $EMAIL_SEND -eq 1 ]); then
echo "emailing results..."
mail -s "$EMAIL_SUBJ_SUCCESS" $EMAIL_ADDR < $LOG_FILE
echo "email sent"
echo ""
else
echo "emailing FAILURE results..."
mail -s "$EMAIL_SUBJ_FAILURE" $EMAIL_ADDR < $LOG_FILE
echo "email ($EMAIL_ADDR) sent"
echo ""
fi

#
# delete log file
#
rm -f $LOG_FILE

exit 0


BASH_ROUTINES.INCL:

##############################################################################
#
# BASH_ROUTINES.INCL
# various bash functions
# release 0.90
#
# copyright (C) 2006 business learning incorporated
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
#
##############################################################################

##############################################################################
#
# parse_args() arg_to_test arg_list_array correct_usage_str
#
# returns: 0 / 1 (no/bad argument found)
# depends: none
#
##############################################################################
parse_args()
{
local ARG_LIST="$3"
local BAD_ARG=1
local PASSED_ARRAY

PASSED_ARRAY=(`echo "$2"`)

for element in "${PASSED_ARRAY[@]}"
do
if [ "$1" = "$element" ]; then BAD_ARG=0; fi
if [ ! "$ARG_LIST" = "" ]; then ARG_LIST="${ARG_LIST}|"; fi
ARG_LIST="${ARG_LIST}${element}"
done

echo "${ARG_LIST[@]}"
return "$BAD_ARG"
}

##############################################################################
#
# check_arg_syntax() arg_1 arg_2
#
# returns: none / exit 1
# depends: parse_args()
#
##############################################################################
check_arg_syntax()
{
local BAD_ARG_1=0
local BAD_ARG_2=0
local NAMES_LIST=""
local INTERVAL_LIST=""

ARRAY_TO_PASS=`echo ${SNAPSHOT_NAME_ARGS[@]}`
NAMES_LIST=(`parse_args "$1" "$ARRAY_TO_PASS" "$NAMES_LIST"`)
BAD_ARG_1="$?"

ARRAY_TO_PASS=`echo ${SNAPSHOT_INTERVAL_ARGS[@]}`
INTERVAL_LIST=(`parse_args "$2" "$ARRAY_TO_PASS" "$INTERVAL_LIST"`)
BAD_ARG_2="$?"

if ([ $BAD_ARG_1 -eq 1 ] || [ $BAD_ARG_2 -eq 1 ]); then
echo ""
echo "usage: ${0} ${NAMES_LIST} ${INTERVAL_LIST}"
echo ""
exit 1
fi
}

##############################################################################
#
# check_for_root_user()
#
# returns: none / exit 1
# depends: none
#
##############################################################################
check_for_root_user()
{
if [ `id -u` != "0" ]; then
echo ""
echo "root privileges required"
echo ""
exit 1
fi
}

.

Relevant Pages

  • Re: [Full-disclosure] reduction of brute force login attempts via SSHthrough iptables --
    ... Anyhow its no problem at all to modify, so if you dont like it, just dont use it. ... on a lame script like this as long as it WORKS and is not insecure. ... echo "~ sorting out ip by ip" ... # echo "not enough failed logins, probably no attack from: ...
    (Full-Disclosure)
  • Ripping Tapes with Linux--How To
    ... the full script is located in the BASH ... My tape player is a portable style journalist's recorder from the ... The most irritating part of the recording process was setting the ... echo Starting de-noise procedure to file $TMP1 ...
    (comp.os.linux.misc)
  • Re: Command Script variable value lost during execution
    ... The value of RC_ver is numeric but ECHO would return it numeric or not... ... I checked ERRORLEVEL at the end of the script and it was 0 but again ... ECHO RoboCopy version xxx%RC_Ver%yyy ... CALL:GetRCVer RoboCopy.exe /Path ...
    (microsoft.public.windows.server.scripting)
  • Re: Bit Twister: Is this the dhclient-exit-hooks you were talking about?
    ... If you are running the dhclient then all you have to do is ... create the hooks script with YOUR CODE. ... DNS server with my wgets and then restart my firewall. ... You can put an echo statement on commands to show you what will happen ...
    (alt.os.linux)