Before you try any backups from the NAS4Free system, you may wish to see what files and directories will actually be backed up from each system on your network. After you've installed DeltaCopy on Windoze, or rsyncd on your Linux boxes, you can use this command from the console on the NAS4Free system or a logged-in terminal to see what rsync modules are available for backup:
/usr/local/bin/rsync rsync://mysys
For each of the modules listed, you can verify what files and directories will be backed up by passing the individual module names to rsync, one at a time, like this:
/usr/local/bin/rsync --recursive --list-only rsync://mysys/modname
Once everything looks good, the following script can be used to back up files from Windoze or Linux remote systems using rsync. Copy it to your System share and then, using a copy command from the console or a logged-in terminal, move it to the OS volume, where nobody will monkey with it:
su mkdir /etc/rsync cp /mnt/System/backup /etc/rsync chown root:wheel /etc/rsync/backup chmod u=rwx,go=rx /etc/rsync/backup
Here is the backup script. You should change the email variables, FromAddr and ToAddr, before you install the script. You may also wish to change the defalut ZFS pool, if you don't plan on setting it via the command line. And, the default backup file ownership and permissions should be set to something other than "root:wheel", etc., if you don't like those settings and don't wish to set them for each backed up directory, in the config file.
/etc/rsync/backup:
#!/bin/bash # # Shell script to back up files from remote systems, using rsync, to the # ZFS pool on NAS4Free. # # This script should be run at regular intervals (typically hourly), by # cron, under the root userid. It can be run whenever backups should be # done with the times set in the config file determining when the backups # are actually done. The command line looks like this: # # backup [-d|--debug] [-h|--help] [-n|--noemail] [-v|--verbose] # configfile [zfspool] # # The parameters are: # # -d|--debug Set this debug flag to test the config file, # etc. Doesn't actually run rsync. # # -h|--help Setting this flag displays help information. # # -n|--noemail If this flag is set, don't send an e-mail with # the results. Normally, if the email variables # are set (below), email is sent to the specified # address, giving the results of the backup. # # -v|--verbose If this flag is present, output status # information to stdout, as the backup # progresses. # # configfile The configuration file that drives the backups # to be done (see below). # # zfspool Optional ZFS pool name, if you wish to # override the internal value (set below). # # The configuration file that is read by this backup script is used to # decide which Rsync modules (share points) on one or more remote systems # should be backed up. # # Note that the configuration file must not contain any of the evil, # MS-generated carriage returns. If it does, this script will undoubtedly # function incorrectly. This ain't f**king Windoze, Sparky. # # The config file contains one or more source systems, specified by the # "Source" keyword, followed by an equal sign and the system's name or IP # address. If a machine name is given, it must be able to be resolved via # DNS or the hostname must be added to the /etc/hosts file via the # Network/Hosts tab on the NAS4Free Web page. # # After that, you may supply one or more destination directories (in the # ZFS pool) and one or more time strings. These are specified by the "Dest" # and "Time" keywords respectively, followed by an equal sign and the # destination directory or time string. # # The destination directory is given as a relative subdirectory underneath # the ZFS pool that is either implied or specified on the command line. # If the directory doesn't exist in the ZFS pool, it will be created. # # The time string is given as one or more hours, separated by spaces, in a # simple list. The hour numbers use the 24-hour clock. If this backup # script is run at an hour that matches one of those in the list, the # backup will be done. A time string of "" will match all backup times. # # The ownership and permissions of the backed up directories and files will # be set to the default values set within this script. If you don't like # this arrangement (e.g. if you are trying to adhere to a more diverse # security scheme), you may set the permissions on an individual backup # directory basis using the BackupOwner, DirPerms and FilePerms parameters, # which must occur before the module list for the directory that they are to # apply to. # # The values for these parameters must comply with the rules for the chown # and chmod commands. The ownership parameter will be applied to both the # directories and files that are backed up. The DirPerms parameter will # give the permissions only for directories. The FilePerms parameter will # give the permissions only for files. Sample values might be: "root:wheel"; # "u=rwx,go=rx" for directories; and "u=rw,go=r" for files. # # Following the keywords, lists of Rsync modules to be backed up to the # backup directory are given to direct the actual backups. The list can # consist of the string "", in which case all of the Rsync modules found # on the remote system will be backed up. If the list begins with a minus # sign, followed by a space, all of the Rsync modules found on the remote # system will be backed up, except for those modules listed. Otherwise, # only those modules listed will be backed up. The list of modules should # be separated by blanks. # Blank lines and lines that begin with "" can be liberally interspersed # throughout the config file. Comments may not be applied to the parameter or list lines, even if they do start with "". Note that the keywords # accumulate and the current value for all three is used whenever a backup # list is found. # # Here is a sample config file: # # # Backup the Profiles directory on Audrey. # Source = audrey # # Dest = Backup_Audrey # Time = 0 # Profiles # # # Backup all of the directories on Gabriella. # Source = gabriella # # Dest = Backup_Gabby # Time = 0 # BackupOwner = root:gabby # DirPerms = ug=rwx,o= # FilePerms = ug=rw,o= # * # # # Backup all of the directories on Video-Game (at diffent times). # Source = video-game # # Dest = Backup_Windoze # Time = 2,10,14,18 # - Profiles # # Dest = Backup_Video-Game # Time = 0 # Profiles # # Note that this script creates a lock file for each destination directory # to prevent multiple backups from running at the same time. If the system # crashes while the lock is held or you need to break the lock for any other # reason, simply delete the file "/var/run/rsync_client_running_backupdir", # replacing the string "backupdir" with the actual name of the top level # backup directory. # ############################################################################ # # Default ZFS pool where all backup files are to be stored. You can modify # this parameter to reflect your NAS4Free configuration, if you don't use the # zfspool positional on the command line. # DefZFSPool="/mnt/Pool1/Datastore1" # # File ownership. Rsync has its own ideas about who should own what. We # may have different ideas. # DefBackupOwner="root:wheel" DefDirPerms="u=rwx,go=rx" DefFilePerms="u=rw,go=r" # # Email variables. Set these up, if you want to have the script email you # backup status. # FromAddr=NAS4Free.Alexandria@gntrains.com ToAddr=ewilde@gntrains.com # # Email commands. # Printf=/usr/bin/printf MailerSMTP=/usr/local/bin/msmtp MailerSMTPConf=/var/etc/msmtp.conf # # Arguments. # Debug=0 WantMail=1 Verbose=0 # # Work areas. # Body="" Backups=0 Errors=0 LineNum=0 DestDir="" SourceSys="" TimeStr="" ############################################################################ # # Function to do an rsync backup from the source system to the destination # directory. # function BackupSource { # # Make sure that the user has indicated the source system whose files # are to be backed up. # if [ x"$SourceSys" = x ]; then if [ $Debug = 1 ]; then echo " No source host specified for backup" else let Errors=Errors+1 LogMsg "No source host specified for backup at config file \ line $LineNum" fi return fi # # Make sure that the user has specified the destination # directory where backups are to be put. # if [ x"$DestDir" = x ]; then if [ $Debug = 1 ]; then echo " No destination specified for backup" else let Errors=Errors+1 LogMsg "No destination specified for backup at config file \ line $LineNum" fi return fi # # If we aren't debugging the config file, we should check that the # time we were given is now, before we do any work. Otherwise, there's # nothing to do. # if [ x"$TimeStr" != x ] && [ "$TimeStr" != "*" ] && [ $Debug != 1 ]; then # # See if the current hour is in the run time string. # CurrHour=$CurrentHour for RunTime in $TimeStr; do if [ $RunTime = $CurrHour ]; then CurrHour="" break fi done # # If the current hour wasn't found in the list, we're done. # if [ x"$CurrHour" != x ]; then return fi fi # # If the module list that we were given is one that requires the full # list of modules (share points) from the remote system, or one that # operates on the full list, let's get it from the remote system now. # if [ "$1" = "" ] || [ "$1" = "-" ]; then SourceModules=`/usr/local/bin/rsync rsync://${SourceSys}` # # If the list isn't the full list, but one that we modify, let's # do that now. # if [ "$1" = "-" ]; then shift 1 IncludedModules="" # # Compare the list from the source against the exclude list. # for Module in $SourceModules; do for Exclude in "$"; do if [ "$Module" = "$Exclude" ]; then Module="" break fi done # # This particular module isn't excluded so add it to the # list. # if [ x"$Module" != x ]; then if [ x"$IncludedModules" = x ]; then IncludedModules=$Module else IncludedModules="${IncludedModules} ${Module}" fi fi done SourceModules=$IncludedModules fi else SourceModules="$*" fi # # If we're debugging the config file, we now have the list of modules # to be backed up so we can dump that list and get out. # if [ $Debug = 1 ]; then echo "Backing up \"$SourceModules\"" return fi let Backups=Backups+1 # # If the destination directory doesn't exist, create it now. # if [ ! -d ${ZFSPool}/${DestDir} ]; then mkdir ${ZFSPool}/${DestDir} chown $BackupOwner ${ZFSPool}/${DestDir} chmod $DirPerms ${ZFSPool}/${DestDir} fi # # Log the beginning of our work. # LogMsg "Starting rsync backup from ${SourceSys} to ${DestDir}" # # Check if we're already backing up the selected modules. # if [ -r /var/run/rsync_client_running_${DestDir} ]; then LogMsg " Previous backup still running, skipping this one" return fi # # Set the lock file. # /usr/bin/touch /var/run/rsync_client_running_${DestDir} # # Copy all of the modules on the remote system. # for Module in $SourceModules; do LogMsg " Backing up module ${Module}" /usr/local/bin/rsync --log-file=/var/log/rsync_client.log \ --recursive --times --compress --archive \ --delete --delete-delay --quiet --perms \ "rsync://${SourceSys}/${Module}" \ "${ZFSPool}/${DestDir}/${Module}" RetCode=$? if [ $RetCode != 0 ]; then let Errors=Errors+1 LogMsg " Backup failed with return code: $RetCode" fi # # Set the ownership and permissions. # # We start by setting the file permissions on everything. This is # pretty fast, since we use the "-R" option on chmod. Then, we # come back and set the directory permissions using find, which is # also pretty fast, since it only works on the directories. Note # that this works, even if the file permissions take away the # execute (i.e. list) permission of the directory because the # recursive find will add the execute permission back to the # directory before it dives into it. # chown -R $BackupOwner ${ZFSPool}/${DestDir}/ chmod -R $FilePerms ${ZFSPool}/${DestDir}/ find ${ZFSPool}/${DestDir} -type d -exec chmod $DirPerms \{\} \; done # # Clear the lock file. # /bin/rm -f /var/run/rsync_client_running_${DestDir} # # Log the end of our work. # LogMsg "Completed backup from ${SourceSys} to ${DestDir}" } ############################################################################ # # Function to add a message to the email body and/or log it to stdout. # function LogMsg { # # Add the message to the email body. # if [ $WantMail = 1 ]; then Body="${Body}${1}\n" fi # # Output the message to stdout, if in verbose mode. # if [ $Verbose = 1 ]; then echo $1 fi } ############################################################################ # # Function to send an email status message, if the user has requested it. # function SendMail { if [ $WantMail = 1 ]; then # # Create a subject. # Subject="NAS4Free rsync backup - `date +"%Y %b %d %H:%M:%S"`" if [ $1 -gt 0 ]; then Subject="${Subject}: Error Detected" fi # # Send the message. # $Printf "From:$FromAddr\nTo:$ToAddr\nSubject:$Subject\n\n$Body" | \ $MailerSMTP --file=$MailerSMTPConf -t fi } ############################################################################ # # Process all of the arguments (flags and positionals). # Args=("$*") for Arg in $Args; do case $Arg in # # Flags. # "-d" | "--debug") Debug=1 ;; "-n" | "--noemail") WantMail=0 ;; "-v" | "--verbose") Verbose=1 ;; "-h" | "--help") echo <<-ENDHELP usage: $0 [-d|--debug] [-h|--help] [-n|--noemail] [-v|--verbose] configfile [zfspool] -d|--debug Debug the config file but don't run rsync. -h|--help Display this help. -n|--noemail Don't send an e-mail with the results. -v|--verbose Output status information to stdout. configfile Config file with backup tasks defined therein. zfspool ZFS pool name, if iternal name is not acceptable. ENDHELP exit ;; # # Positionals. # *) if [ x"$ConfigFile" = x ]; then ConfigFile=$Arg else if [ x"$ZFSPool" = x ]; then ZFSPool=$Arg fi fi ;; esac
done
#
# If there are no email addresses, we'll turn off email.
#
if [ x"$FromAddr" = x ] || [ x"$ToAddr" = x ]; then
WantMail=0
fi
#
# Make sure that the user has indicated where the config file is.
#
if [ x"$ConfigFile" = x ]; then
Verbose=1 LogMsg "No config file specified listing the backup tasks" SendMail 1 exit 1
fi
#
# See if the user has specified the ZFS pool where backups are to be put.
#
if [ x"$ZFSPool" = x ]; then
ZFSPool=$DefZFSPool
fi
#
# Set the default ownership and permissions.
#
BackupOwner=$DefBackupOwner
DirPerms=$DefDirPerms
FilePerms=$DefFilePerms
#
# Get the current hour. Used to compare against the time string parameter
# in the config file.
#
CurrentHour=`date +%k`
#
# Read the config file and process all of the backups therein.
#
while read Line; do
# # Count lines for error reportage. # let LineNum=LineNum+1 if [ $Debug = 1 ]; then echo Line $LineNum: $Line fi # # If the line is not a comment, process it. if [ "`echo $Line | grep '^[^]'`"x != x ]; then # # If the line is a parameter and its value, process it. The # parameter will be turned into a shell variable. Here's what we # are looking for: # # BackupOwner The ownership string to apply to the backed up # directories and files (e.g. root:wheel). # # DestDir The name of the top level directory in the # ZFS pool where the remote system's files are # to be backed up. # # DirPerms The directory permission string (e.g. # u=rwx,go=rx). Only applies to backed up # directories. # # FilePerms The file permission string (e.g. u=rw,go=r). # Only applies to backed up files. # # SourceSys The source remote system whose files are to # be backed up. This is given as either an IP # address or a machine name. # # TimeStr The time string, containing a list of all # times when the backup should be done. Only # the hours are given, in 24-hour format, and # separated from one another by spaces. # # Note that you are supposed to be able to turn values read into # variables into actual shell variables like this: # # eval ${ParmName}=`echo -ne \""$ParmVal"\"` # # This doesn't work on the NAS4Free version of bash. Furthermore, # the shell variable name that is set depends on what name the # user uses in the config file. # # Instead, we look for what we want and set it. This lets us # control what we get and give error feedback if it ain't what we # want. # if [ "`echo $Line | grep -E '^[^=]+=[^=]+$'`"x != x ]; then # # Pick up the parameter name and value. Note that the two # values in the square brackets of the sed lines (below) are # <space> and <tab>. You cannot use "\t", "\\t", "\\\\t", # "\\\\\\t" or any other combination of backslashes and the # letter "t" that you might think of. Too baddie! You need # to use a real, hard tab. # ParmName=`echo "$Line" | awk -F"=" '{print $1}' | sed "s/[ ]\$//"` ParmVal=`echo "$Line" | awk -F"=" '{print $2}' | sed "s/^[ ]//"` if [ $Debug = 1 ]; then echo " Parm $ParmName = \"$ParmVal\"" fi # # Look for all of our variables. # case "$ParmName" in # # Backup owner. # backupowner|owner|BackUpOwner|Owner) BackupOwner=$ParmVal ;; # # Destination. # dest|destination|Dest|Destination) DestDir=$ParmVal ;; # # Directory permissions. # dirperms|DirPerms) DirPerms=$ParmVal ;; # # File permissions. # fileperms|FilePerms) FilePerms=$ParmVal ;; # # Source. Note that this parameter resets all of the # other parameters so that stuff left over from the # previous source isn't used with this source. # source|Source) SourceSys=$ParmVal BackupOwner=$DefBackupOwner DestDir="" DirPerms=$DefDirPerms FilePerms=$DefFilePerms TimeStr="" ;; # # Time string. # time|timestr|Time|Timestr|TimeStr) TimeStr="$ParmVal" ;; # # Unknown parameter. # ) if [ $Debug = 1 ]; then echo " Invalid parameter $ParmName" else let Errors=Errors+1 LogMsg "Invalid parameter $ParmName in config file \ at line $LineNum" fi esac # # Otherwise, the line is treated is a list, unless it is blank. # else if [ "`echo $Line | grep -c '^\\s$'`" = 0 ]; then if [ $Debug = 1 ]; then echo " List = \"$Line\"" fi # # Looks like we're good to go. # if [ "$Line" = "" ]; then BackupSource "" else BackupSource $Line fi fi fi fi
done < $ConfigFile
#
# Give a report of the number of errors.
#
if [ $Errors -gt 0 ]; then
LogMsg "$Errors error(s) occurred during backup"
fi
#
# Send email, if required.
#
if [ $Errors -gt 0 ] || [ $Backups -gt 0 ] ; then
SendMail $Errors
fi
Before you run this script, you'll need to create a config file to tell it which sources and modules (share points) are to be backed up and where the backups are to be stored. The script itself has quite a bit of documentation about how to do this so we'll just show a sample config file here. We put it in the System share and then, using a copy command from the console or a logged-in terminal, move it to the OS volume, where nobody will monkey with it:
su cp /mnt/System/backup.cfg /etc/rsync chown root:wheel /etc/rsync/backup.cfg chmod u=rwx,go=rx /etc/rsync/backup.cfg
/etc/rsync/backup.cfg:
# This configuration file is read by the NAS4Free backup script to decide # which Rsync modules (share points) on one or more remote systems should # be backed up. # # Note that this file must not contain any of the evil, MS-generated # carriage returns. This ain't f**king Windoze, Sparky. # Backup all of the directories on Audrey. Source = audrey Dest = Backup_Audrey Time = 0 * # Backup all of the directories on Video-Game. Source = video-game Dest = Backup_Windoze Time = 2 10 14 18 - Profiles Dest = Backup_Video-Game Time = 0 Profiles # Backup the global directories on the gateway server. Source = gateway Dest = Backup_Gateway Time = 0 mail mysql # Backup user directories on the gateway server. Source = gateway Dest = Backup_HomeDirs Time = 2 10 14 18 home-jsmith home-jblow
Before you do any actual damage, you can test your config file by running the script with the debug flag:
/etc/rsync/backup -d /etc/rsync/backup.cfg
Once you're happy with how your config file is being handled, you can test the operation of the script like this:
su /etc/rsync/backup -n -v /etc/rsync/backup.cfg
If any of the backups specified in the config file occur at the current time (i.e. the current hour), you should see the results of the backup on your terminal. If the time isn't correct, you can come back later or just modify the config file to use the current times. And, if you want to test that the backup job's status message is being delivered properly, you can test it like this:
su /etc/rsync/backup /etc/rsync/backup.cfg
When you are happy with the script and the config file, you should add a cron table entry to run the backup script at hourly intervals (the config file will decide which backups actually run during each hour).
System/Advanced/Cron
Begin by clicking the "Plus" button to add a new entry. Command: /etc/rsync/backup /etc/rsync/backup.cfg Who: root Description: Back up source_host on a regular basis. Schedule time: Minutes - 0, Hours - All, Days - All, Months - All, Weekdays - All If you want to test the script, you can click the "Run now" button. This is probably a good idea because the cron job may fail silently later on. Click the "Add" button to save the crontab entry and then click to "Apply Changes" button to actually save the crontab entry.