Scrubbing

One of the benefits of using ZFS is the ability to scrub the data, that is regularly check it to see that its integrity is intact by comparing mirrored copies and checksums. If an error that is caused by bit-rot is found, the scrub can correct it before it becomes permanent.

For commercial-grade disk drives, monthly scrubbing is recommended. For consumer-grade disk drives, weekly scrubbing is recommended. Since scrubbing is an expensive operation (all data on the ZFS pools must be checked), one must consider how often to schedule the scrub. Personally, we prefer to do it once a week, regardless of the type of storage. Sometime in the wee hours on an off day is preferred.

The following script, taken from gimpe, with a couple of small corrections, will perform scrubs in the background. Copy it to your System share and then, using a copy command from the console or a logged-in terminal, move it to the OS volume, where nobody will monkey with it:

     su
     cp /mnt/System/scrub /etc/zfs
     chown root:wheel /etc/zfs/scrub
     chmod u=rwx,go=rx /etc/zfs/scrub

You can test it like this:

     su
     /etc/zfs/scrub -n -v

You should see the results of the scrub on your terminal. If you want to test that the status message is being delivered properly, you can test it like this:

     /etc/zfs/scrub

Here is the scrub script. You should change the email variables, FROM, TO, and possibly SUBJECT, before you install the script:

/etc/zfs/scrub:

     #!/bin/bash
     #VERSION: 0.2
     #AUTHOR: gimpe
     #EMAIL: gimpe [at] hype-o-thetic.com
     #WEBSITE: http://hype-o-thetic.com
     #DESCRIPTION: Created on FreeNAS 0.7RC1 (Sardaukar)
     # This script will start a scrub on each ZFS pool (one at a time) and
     # will send an e-mail or display the result when everyting is completed.
     #CHANGELOG
     # 0.2: 2009-08-27 Code clean up
     # 0.1: 2009-08-25 Make it work
     #SOURCES:
     # http://aspiringsysadmin.com/blog/2007/06/07/
     #        scrub-your-zfs-file-systems-regularly/
     # http://www.sun.com/bigadmin/scripts/sunScripts/zfs_completion.bash.txt
     # http://www.packetwatch.net/documents/guides/2009073001.php
     # e-mail variables
     FROM=from@devnull.com
     TO=to@devnull.com
     SUBJECT="Scrub of Pools on NAS4Free"
     BODY=""
     # arguments
     VERBOSE=0
     SENDEMAIL=1
     args=("$*")
     for arg in $args; do
         case $arg in
             "-v" | "--verbose")
                 VERBOSE=1
                 ;;
             "-n" | "--noemail")
                 SENDEMAIL=0
                 ;;
             "-a" | "--author")
                 echo "by gimpe at hype-o-thetic.com"
                 exit
                 ;;
             "-h" | "--help" | *)
                 echo "
     usage: $0 [-v --verbose|-n --noemail]
         -v --verbose    output display
         -n --noemail    don't send an e-mail with result
         -a --author     display author info (by gimpe at hype-o-thetic.com)
         -h --help       display this help
     "
                 exit
                 ;;
         esac
     done
     # work variables
     ERROR=0
     SEP=" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "
     RUNNING=1
     # commands & configuration
     ZPOOL=/sbin/zpool
     PRINTF=/usr/bin/printf
     MSMTP=/usr/local/bin/msmtp
     MSMTPCONF=/var/etc/msmtp.conf
     # print a message
     function _log {
         DATE="`date +"%Y-%m-%d %H:%M:%S"`"
         # add message to e-mail body
         BODY="${BODY}$DATE: $1\n"
      # output to console if verbose mode
      if [ $VERBOSE = 1 ]; then
          echo "$DATE: $1"
      fi

}

     # find all pools
     pools=$($ZPOOL list -H -o name)
     # for each pool
     for pool in $pools; do
         # start scrub for $pool
         _log "starting scrub on $pool"
         zpool scrub $pool
         RUNNING=1
         # wait until scrub for $pool has finished running
         while [ $RUNNING = 1 ];     do
             # still running?
             if $ZPOOL status -v $pool | grep -q "scrub in progress"; then
                 sleep 60
             # not running
             else
                 # finished with this pool, exit
                 _log "scrub ended on $pool"
                 _log "`$ZPOOL status -v $pool`"
                 _log "$SEP"
                 RUNNING=0
                 # check for errors
                 if ! $ZPOOL status -v $pool | grep -q "No known data errors"; then
                     _log "data errors detected on $pool"
                     ERROR=1
                 fi
             fi
         done
     done
     # change e-mail subject if there was error
     if [ $ERROR = 1 ]; then
         SUBJECT="${SUBJECT}: Error(s) Detected"
     fi
     # send e-mail
     if [ $SENDEMAIL = 1 ]; then
         $PRINTF "From:$FROM\nTo:$TO\nSubject:$SUBJECT\n\n$BODY" | \
                 $MSMTP --file=$MSMTPCONF -t
     fi

Add a cron table entry to run the scrub script at regular intervals by adding an entry on the Cron page.

System/Advanced/Cron

     Begin by clicking the "Plus" button to add a new entry.
     Command: /etc/zfs/scrub
     Who: root
     Description: Scrub the ZFS pools on a regular basis.
     Schedule time: Minutes - 0, Hours - 1, Days - All, Months - All,
                    Weekdays - Sunday
     If you want to test the script, you can click the "Run now" button.  This
     is probably a good idea because the cron job may fail silently later on.
     Click the "Add" button to save the crontab entry and then click to "Apply
     Changes" button to actually save the crontab entry.