UPS Monitoring with NUT

The quick command to set the UPS battery date, after changing batteries is:

     /bin/upsrw -s battery.date=mm/dd/yy -u username -p password \
       upsname@localhost

This presupposes that the username and password are set in upsd.users and that "actions = set" are specified for the user. Also, upsd and the appropriate UPS driver must be up and running for the UPS in question.

If you are running a version of NUT that does not set the battery.date variable correctly for the APC SmartUPS, you can do it manually.

First you must make sure that you have a copy of "screen" installed on the system in question (use "apt-get install screen"). You must also make sure that the UPS driver is not running. Then, assuming that you have the UPS connected to /dev/ttyS0, do the following (note that there are no carriage returns used for any of the commands sent to the UPS):

     su
     /etc/init.d/nut stop
     screen /dev/ttyS0 2400,cs8
     Y
     (UPS echoes "SM")
     x
     (UPS echoes current date)
     -
     (UPS echoes nothing, possibly including the "-", i.e. really nothing)
     mm/dd/yy
     (after about 5 seconds, UPS echoes "|", indicating EEPROM is changed;
      some versions echo "OK", instead)
     x
     (UPS echoes the new date)
     R
     (UPS echoes "BYE")
     <Ctrl-a>\
     (answer yes to exit screen)
     /etc/init.d/nut start

Similarly, if you are running a version of NUT that does not set the UPS identifier correctly for the APC SmartUPS, you can set an up-to-eight-character identifier manually, using "screen" and assuming that you have the UPS connected to /dev/ttyS0, as follows:

     su
     /etc/init.d/nut stop
     screen /dev/ttyS0 2400,cs8
     Y
     (UPS echoes "SM")
     c
     (UPS echoes current UPS ID, probably UPS_IDEN for a new UPS)
     -
     (UPS echoes nothing, possibly including the "-", i.e. really nothing)
     upsname<Enter>
     (after about 5 seconds, UPS echoes "|", indicating EEPROM is changed;
      some versions echo "OK", instead)
     (on some UPS, you may have to press <Enter> twice)
     c
     (UPS echoes the new UPS ID)
     R
     (UPS echoes "BYE")
     <Ctrl-a>\
     (answer yes to exit screen)
     /etc/init.d/nut start

A full description of the APC SmartUPS protocol can be found at:

     http://www.apcupsd.com/manual/manual.html

Also, if you want to get rid of the "Change the battery" message and the red light on the UPS panel, you can force the battery test to be rerun immediately. However, before you do this, wait for three or four hours after the new battery is installed to give the UPS time to charge it first. Then, run the battery test, which will reset everything, if the new battery is OK. To do this, use:

     /bin/upscmd -u username -p password upsname@localhost test.battery.start

The battery test should run (you'll see all of the messages it usually generates) for a few seconds and all will be well.

To do a complete install, begin by installing the latest version of NUT using the Package Manager or Ubuntu Software Center. You will need these packages (plus any dependencies):

     nut
     nut-cgi

You can also install these packages using apt-get, as follows:

     sudo apt-get install nut
     sudo apt-get install nut-cgi

Since Ubuntu uses udev, we need to hack one of the udev rules to give permission on the appropriate serial port. The best place is in a separate rules file that you make up just for this purpose (that way, subsequent releases of NUT won't monkey with your rules for the serial port). We suggest calling the file "52_nut-ttyups.rules. It should be created in the /etc/udev/rules.d directory. You should add something like the following to that file, depending on which serial port you need to use:

     # udev rules for NUT serial drivers
     # Special case for the UPS devices
     KERNEL=="ttyS1", GROUP="nut", MODE="0660"

/etc/nut/ups.conf:

     Copy the file ups.conf.sample and hack it to set up the machine/UPS
     configuration, per the instructions in the INSTALL file.  For an APC
     SmartUPS, the following config information applies:
     [fusion-reactor]
          driver = apcsmart
          port = /dev/ttyS0

/etc/nut/upsd.conf:

     On older versions of NUT, copy the file upsd.conf.sample and replace the
     existing rules with:
     ACL all 0.0.0.0/0
     ACL localhost 127.0.0.1/32
     ACL homeworld 192.168.1.0/24
     ACCEPT homeworld
     ACCEPT localhost
     REJECT all
     On newer versions of NUT (e.g. the version that comes with Mythbuntu 9.04),
     copy the upsd.conf.sample file and replace the LISTEN directives with:
     LISTEN 127.0.0.1
     LISTEN 192.168.1.123 3493
     You should use the actual IP address of the machine where NUT is being
     installed in place of 192.168.1.123 (above).  If the machine has two or more
     NICs and you want NUT to listen on all of them, add additional LISTEN
     directives for the IP address of each of them.
     Note that, if you have an older config file and are upgrading to a newer
     version of NUT, you will need to remove all of the ACL, ACCEPT and REJECT
     rules and simply use LISTEN.  Apparently, the designers have decided that NUT
     should get out of the security business and leave everything to the firewall.
     Consequently, the rules are no longer accepted and LISTEN is simply used to
     tell NUT which port to listen on.

/etc/nut/upsd.users:

     Copy the file upsd.users.sample. Add a user that will allow upsmon on the
     local host to shut down the machine if the power goes south as well as do
     periodic battery tests:
     [lmmonitor]
         password = ItsASecret
         allowfrom = localhost
         instcmds = test.battery.start
         actions = set
         upsmon master

/etc/nut/upsmon.conf:

     Copy the file upsmon.conf.sample. Change the group ownership to the nut user
     and give it group permissions:
     chgrp ups /etc/nut/upsmon.conf
     chmod g+rw /etc/nut/upsmon.conf
     Make the following changes to the file to begin monitoring the UPS:
     NOTIFYCMD /etc/nut/notify
     RUN_AS_USER nut
     MONITOR fusion-reactor@localhost 1 lmmonitor ItsASecret master
     NOTIFYFLAG COMMBAD  SYSLOG+EXEC
     NOTIFYFLAG COMMOK   SYSLOG+EXEC
     NOTIFYFLAG FSD      SYSLOG+EXEC
     NOTIFYFLAG LOWBATT  SYSLOG+EXEC
     NOTIFYFLAG NOCOMM   SYSLOG+EXEC
     NOTIFYFLAG ONBATT   SYSLOG+WALL+EXEC
     NOTIFYFLAG ONLINE   SYSLOG+WALL+EXEC
     NOTIFYFLAG REPLBATT SYSLOG+EXEC
     NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
     NOTIFYFLAG OVERTEMP SYSLOG+EXEC
     If you want to monitor UPS temperature and you've applied the proper hacks
     (or the temperature hacks are included in your version of the source), add
     the following:
     # --------------------------------------------------------------------------
     # UPSOVERTEMP - Temperature (in Celcius) which is too high for operation
     #
     # upsmon will check all UPS that return temperature information against
     # this value.  If the UPS temperature exceeds this value, an OVERTEMP
     # notification will be generated.
     #
     # Note that certain UPS are renown for cooking and even burning up
     # batteries (some reports of spectacular battery fires have been received).
     # From actual observed log data, it appears that prior to burning up the
     # batteries, the UPS internal temperature rises significantly.  Hence,
     # monitoring the UPS temperature can be a valuable tool towards detecting
     # battery cooking, before the UPS burns the place down (the UPS is supposed
     # to solve problems, not cause them, isn't it).
     #
     # Once again, typical observed internal temperatures are in the 40 to 50
     # degree Celcius range.  Observed temperatures of 80 degrees Celcius prior
     # to an actual battery failure are indicative of pending failure.  Thus, to
     # be safe, the the UPSOVERTEMP value should be set in the 60-70 degree
     # range.
     UPSOVERTEMP 60.0

/etc/nut/notify:

     Create the following script to send event notifications to root, via email,
     whenever the UPS has something important to say.  Note that the indentation
     in front of "ENDMSG" can be tabs only.  If your lame-ass text editor sticks
     spaces in there, the shell script will get a syntax error.  Set the
     UPS_BOXES variable to your list of UPS boxes.  Here is the script:
     #!/bin/sh
     # A shell script that can be used by the UPS power monitor to send messages
     # to root when power failures occur.
     #
     # If you would like to log all of the UPS events that take place in the log
     # file too, define the location of the log file.  Otherwise, if the
     # LOG_PATH is set to an empty string, no log entries are written.
     # Define the log path and the boxes we're logging.
     LOG_PATH="/var/log/upslog"
     UPS_BOXES="fusion-reactor bevatron"
     # Write the event to the log, if there is one.
     if [ x"$LOG_PATH" != x ]; then
         timestamp=`date "+%Y/%m/%d %H:%M:%S"`
         for LogBox in $UPS_BOXES; do
             echo $UPSNAME | grep -q $LogBox
             matval=$?
             if [ $matval = 0 ] ; then
                 echo $timestamp EVENT: $1 >> ${LOG_PATH}.$LogBox
             fi
         done
     fi
     # Some events we only log.
     if ([ x"$NOTIFYTYPE" != xONBATT ]) && ([ x"$NOTIFYTYPE" != xONLINE ]) \
         && ([ x"$NOTIFYTYPE" != xREPLBATT ]) \
         && ([ x"$NOTIFYTYPE" != xSHUTDOWN ]) \
         && ([ x"$NOTIFYTYPE" != xOVERTEMP ]); then
         exit 0
     fi
     # Email the message to root so that they can see everything that's going
     # on.  After all, if you're omnipotent, you need to know everything.
     HostName=`hostname`
     if [ x"$NOTIFYTYPE" != xSHUTDOWN ]; then
         /usr/bin/mail -s "Message from the UPS" root <<-ENDMSG
             The power monitor on $HostName has generated the following message:
             $1
             ENDMSG
     else
         /usr/bin/mail -s "Urgent message from the UPS" root <<-ENDMSG
             The power monitor on $HostName has generated the following message:
             $1
             ENDMSG
     fi
     Note that, in the above script, the two sets of lines between
     "/usr/bin/mail -s ... <<-ENDMSG" and up to and including the line with
     "ENDMSG" must either not be indented at all or only indented with actual tab
     characters (not blanks).  If they are not, the script will fail.
     Also, don't forget to add execute permissions to the script after you create
     it:
     su
     chmod ugo+x /etc/nut/notify
     Then, it wouldn't hurt try it out after you've got it all set up.  For
     example:
     UPSNAME=bevatron; export UPSNAME
     NOTIFYTYPE=ONLINE; export NOTIFYTYPE
     /etc/nut/notify "This is a test"
     Check that root receives the message and that the message gets logged to
     bevatron's log file.

/etc/nut/hosts.conf:

     Copy the file hosts.conf.sample.  Add all of the machines that you want to be
     able to monitor from the Web.  Also add the name of the logfile, if logged
     events will be displayed:
     MONITOR fusion-reactor@localhost "pri-host UPS"
     LOGFILE /var/log/upslog.fusion-reactor

/etc/nut/ftplogs:

     If you'd like to build a consolidated graph of all your UPS activity and have
     a centralized server where logfiles can be copied for this purpose, you
     should create the file /etc/nut/ftplogs:
     #! /bin/sh
     #
     # ftplogs - Script to ftp the UPS logs to the primary server at regular
     #           intervals.
     #
     # This script is used both by cron, to send the current UPS logs to the
     # primary server, every 15 minutes, and by logrotate, to send the freshly
     # rotated UPS logs to the primary server whenever UPS logs are rotated.
     #
     #
     # Define the log path and the boxes we're logging.
     #
     PRI_SERVER="pri-host"                   # Primary server name
     LOG_PATH="/var/log/upslog"              # Local log path
     PRI_LOG_PATH="/var/log/upslog"          # Log path on primary server
     UPS_BOXES="bevatron"
     #
     # If we were passed a logfile name, send it to the primary server directly.
     #
     if test x"$1" != x; then
         echo -e "user nut ItsASecret\\nput ${LOG_PATH}.$1 ${PRI_LOG_PATH}.$1" \
             | ftp -n $PRI_SERVER
     #
     # For all of the UPS on this system, send the logs to the primary.
     #
     else
        for LogBox in $UPS_BOXES; do
            echo -e "user nut ItsASecret\\nput ${LOG_PATH}.$LogBox \
                ${PRI_LOG_PATH}.$LogBox" | ftp -n $PRI_SERVER
        done

fi

     Note that on some machines /bin/sh is an alias for /bin/dash and that dash,
     apparently, doesn't have to follow the rules too closely for shells.  So, you
     may find that dash is broken when it comes to echo and "echo -e" doesn't
     work.  If this is the case (First of all, why do we need another half dozen
     shells, anyway?  Isn't two or three already way too many?  And, who the f**k
     gives a rat's butt about POSIX compatibility?  What I giva a f**k about is
     "working", especially old scripts, that used to work fine, "still working".
     So, nice going a**holes.), you'll probably be a lot happier with /bin/bash
     instead of /bin/sh.
     You should pick a user that is valid on the server, use the right password
     for that user and set the server name to the correct name for the central
     server (make sure this server name appears in either /etc/hosts on the local
     Mythbuntu machine or can be resolved through DNS).  Add the script to your
     cron table so that it runs at regular intervals (e.g. every 15 minutes), as
     shown below.
     Also, on that server, you should create a couple of empty log files with the
     correct permissions so that FTP can overwrite them without getting permission
     denied:
     su
     touch /var/log/upslog.bevatron /var/log/upslog.bevatron.1
     chown root:ups /var/log/upslog.bevatron
     chmod g+w /var/log/upslog.bevatron

/etc/logrotate.d/nut:

     Add the new UPS' logfile to the logrotate config file:
     /var/log/upslog.bevatron {
         notifempty
         missingok
         create 0664 root ups
         copytruncate
         postrotate
             echo HEADER: Bevatron >/var/log/upslog.bevatron
             /etc/nut/ftplogs bevatron.1
         endscript
     }

/etc/crontab:

     If you want the UPS batteries to be tested at regular intervals, you can add
     a line to crontab to do that.  Every couple of months is a good interval.
     If you care, schedule the times for the battery test on days when the other
     UPS are not also testing their batteries (you don't want all the machines
     down at once, do you).
     You should also add a line to schedule the push of the logs to a central
     server, if you want to use that feature.
     # Push the UPS logs over to the primary host every 15 minutes.
     05,20,35,50 * * * * root /etc/nut/ftplogs
     # Every two months, have the UPS check its battery.
     00 6 15 1,3,5,7,9,11 * root /bin/upscmd -u lmmonitor -p ItsASecret \
                                 fusion-reactor@localhost test.battery.start
     Note that most UPS will usually come with automatic self-test turned on and
     set to every 14 days.  This means that the UPS will run its battery test
     itself every 14 days, exactly 1209600 seconds after it is turned on.  How
     convenient is that?  If you'd rather the test was done when you decide, in
     your crontab, you should turn off this dubious feature like so:
     /bin/upsrw -s ups.test.interval=0 -u lmmonitor -p ItsASecret \
                fusion-reactor@localhost

/etc/init.d/ups-monitor:

Note that later versions of NUT name the script in init.d "nut" instead of "ups-monitor". Then, they point a symlink of "ups-monitor" at "nut". If you have one of these versions, you should update the "nut" script instead of "ups-monitor".

The installed ups-monitor script does not start logging. If you'd like to use logging, you can add the following:

     upslog_pid=${pid_dir}/upslog.pid
     upslog=/bin/upslog
          .
          .
          .
     start_stop_log () {
       case "$START_UPSLOG" in
         y|Y|yes|YES|Yes)
           case "$1" in
             start)
               timestamp=`date "+%Y/%m/%d %H:%M:%S"`
              for UPSBox in $UPSLOG_BOXES; do
                echo $timestamp EVENT: Starting UPS logging \
                    >> ${UPSLOG_PATH}.$UPSBox
                chgrp $UPSLOG_GROUP ${UPSLOG_PATH}.$UPSBox
                chmod g=rw ${UPSLOG_PATH}.$UPSBox
                $upslog ${UPSBox}@localhost ${UPSLOG_PATH}.$UPSBox \
                  $UPSLOG_INTERVAL \
                  "%TIME @Y/@m/@d @H:@M:@S% %VAR input.voltage% \
                      %VAR output.voltage% %VAR input.frequency% \
                      %VAR battery.charge% %VAR ups.load% [%VAR ups.status%] \
                      %VAR ups.temperature%" \
                  >/dev/null 2>&1
                startval=$?
                if [ $startval = 0 ] ; then
                  touch ${upslog_pid}.$UPSBox
                else
                  return $startval
                fi
              done
              ;;
            stop)
              timestamp=`date "+%Y/%m/%d %H:%M:%S"`
              for UPSBox in $UPSLOG_BOXES; do
                if [ -f /var/lock/subsys/upslog.$UPSBox ]; then
                  LogPID=`ps -eo pid,args | grep upslog | grep $UPSBox \
                      | sed -n 's/^ \([0-9]\).*/\1/p'`
                  if [ x"$LogPID" != x ]; then
                    kill -9 $LogPID
                    stopval=$?
                    echo $timestamp EVENT: Stopping UPS logging \
                        >> ${UPSLOG_PATH}.$UPSBox
                    [ $stopval = 0 ] && rm -f /var/lock/subsys/upslog.$UPSBox
                  fi
                fi
              done
              ;;
          esac
          ;;
        n|N|no|NO|No|*)
          return 1
          ;;
      esac
    }
         .
         .
         .
     case "$1" in
      start)
        log_daemon_msg "Starting $DESC"
        check_var_directory
        start_stop_server start && log_progress_msg "upsd"
        start_stop_client start && log_progress_msg "upsmon"
        start_stop_log start && log_progress_msg "upslog"
        log_end_msg 0
        ;;
      stop)
        log_daemon_msg "Stopping $DESC"
        start_stop_log stop && log_progress_msg "upslog"
        start_stop_client stop && log_progress_msg "upsmon"
        start_stop_server stop && log_progress_msg "upsd"
        log_end_msg 0
        ;;

Once you've made any changes to ups-monitor for logging, if it wasn't already installed by apt-get, install it using the dain-bramaged update-rc.d script. The following should work:

     update-rc.d nut defaults

or maybe:

     update-rc.d ups-monitor defaults

/etc/nut/nut.conf:

     In a never-ending attempt to justify the five or six config files needed by
     this package, they've added another config file (Yeah!) only this time its
     used by the startup script in init.d to tell it which other config files to
     use.  Consequently, if this file is present and is needed by the startup
     script (either nut or ups-monitor, in init.d), you should edit to get NUT
     working because the default value in it disables NUT, out of the box.
     Change MODE to either "netserver" or "standalone" (they both work the same)
     like this:
     MODE=netserver

/etc/default/nut:

     You will have to hack the default settings so that upsd and upsmon are
     started:
     START_UPSD=yes
     START_UPSMON=yes
     If you want to use the logging features added to ups-monitor (above), add
     these lines:
     # start upslog
     START_UPSLOG=yes
     # set upslog specific options. use "man upslog" for more info
     UPSLOG_OPTIONS=""
     # a list of UPS to log, separated by a space
     UPSLOG_BOXES="Bevatron"
     # logging directory (including filename prefix)
     UPSLOG_PATH="/var/log/upslog"
     # the log interval (in seconds)
     UPSLOG_INTERVAL=30
     # group to use for log files
     UPSLOG_GROUP=nut