Fetchmail

Fetchmail can be run at regular intervals on a system that connects through diald or when you just don't want sendmail to accept mail from the outside world (its more secure that way). Typically, one would edit crontab to run fetchmail.

Note that version 8 of RedHat comes with fetchmail 5.9.0 (or something like that) pre-installed. Unfortunately, a new, annoying behaviour is found in that version. Fetchmail will now try to fall back on procmail for mail delivery, if it cannot connect to port 25 on the local machine. This will effectively bypass all virus/spam filtering if sendmail isn't running -- not a desirable outcome.

To fix the problem or if you just want to run the latest version of fetchmail, obtain the latest fetchmail distribution from:

     http://fetchmail.berlios.de/

Build fetchmail, per the instructions in the INSTALL file. Make sure that the "--enable-fallback=no" option is used on the "./configure" command:

     ./configure --enable-fallback=no
     make

Test the build by entering the command:

     ./fetchmail -V

If the results do not say "Fallback MDA: (none)", you have not succeeded in getting rid of the fallback mailer. If they do say "Fallback MDA: (none)", you can install fetchmail in its usual places:

     su
     make install

/etc/mail/fetchmail-poll:

     A script that can be run a regular intervals (e.g. hourly) by cron to poll
     an ISP's server and see if there's any mail to deliver.  This script is
     basically just a front end wrapped around fetchmail-config (see below), which
     is set up to look for fetchmail config files in all the home directories of
     the users defined on the local machine and fetch mail for any of them that
     has a config file:
     #!/bin/sh
     #
     # Script to fetch mail from the ISP's POP server and pass it through
     # sendmail on the local machine.
     #
     # Should be run at regular intervals from cron.  It will find all of the
     # .fetchmailrc files in the tree under /home and fetch the mail for every
     # user that has one.
     #
     #
     # We may be using diald.  By pinging the POP server, we get diald to dial
     # up the PPP link.  The ping may fail but it brings up the link
     #
     ping -c1 -w120 mail.mydomain.com
     #
     # Find all of the .fetchmailrc files belonging to the users.  Fetch their
     # mail and pass it through sendmail.
     #
     # A typical .fetchmailrc file should look as follows:
     #
     #      poll mail.popserver.com proto pop3
     #        user joeblow with password fromkokomo is jblow here
     #
     /etc/mail/fetchmail-config --primary=192.168.1.1 \
         --secondary=pop.mydomain.com,pop.myotherdomain.com \
         /home .fetchmailrc | \
         /usr/bin/fetchmail -a -f - -l 25000000 -t 600 -v

/etc/mail/fetchmail-config:

A script that can be run against a directory tree to look for individual user config files and consolidate them into a single configuration file that can be piped directly to fetchmail to cause it to fetch mail for all of the users. This script is usually invoked out of cron by fetchmail-poll (see above).

     #!/usr/bin/perl
     #
     # Copyright (c) 2007 Eric Wilde
     #
     # This file is used by fetchmail to set up config files.
     #
     #
     # Modifications
     # -------------
     #
     # Date         Programmer     Description
     #
     # 2009 Feb 22  E.Wilde        Fix bogus close of ping handle when used with
     #                             multiple primaries.
     # 2007 Aug 10  E.Wilde        Initial coding.
     #
     #
     # Usage
     # -----
     #
     # fetchmail-config [--Primary=pri1[,pri2,...]] [--Secondary=sec1[,sec2,...]]
     #                  [--Debug] homedir configfile
     #
     # Primary                     The name or names (comma separated) of the
     #                             primary server(s) that are to be used first, if
     #                             they all exist (i.e. are reachable by ping).
     #                             If this option is omitted, no check for primary
     #                             or secondary servers will be done.
     #
     #                             Note that you must be root to use this option
     #                             and you must also specify the Secondary option.
     #
     #                             Also note that, if any of the primary servers
     #                             resolves to the same IP address as the host on
     #                             which fetchmail-config is running, the switch
     #                             is made to the secondary servers.  This allows
     #                             the same fetchmail-config command to be used on
     #                             both the main and redundant machines, if need
     #                             be.
     #
     # Secondary                   The name or names (comma separated) of the
     #                             secondary server(s) that are to be used only if
     #                             one or more of the primary servers does not
     #                             exist (i.e. aren't reachable by ping).  If this
     #                             option is omitted, no check for primary or
     #                             secondary servers will be done.
     #
     # Debug                       Turn on debugging.  Copious information is
     #                             dumped to stdout (you may want to redirect via
     #                             "perl fetchmail-config ... >file").
     #
     # homedir                     The name of the home directory that is to be
     #                             searched for fetchmail configuration files.
     #                             Only the first level subdirectories below this
     #                             directory will be searched.  This name is
     #                             required.  Typically, one would use something
     #                             like "/home".
     #
     # configfile                  The name of the fetchmail configuration files
     #                             that are to be searched for.  Each subdirectory
     #                             below the home directory given is searched for
     #                             a file that matches this name.  If found, it is
     #                             appended to the fetchmail config file being
     #                             constructed.  Typically, one would use a name
     #                             like ".fetchmailrc".
     #
     #
     # Description
     # -----------
     #
     # This program is run against all users defined in a system to find
     # individual fetchmail configuration files.  It consolidates all of the
     # config files together into a single file which can then be piped through
     # fetchmail.  It also does selection of mail server addresses to allow for
     # fetching of mail on redundant systems.
     #
     # To use it for straight fetching of mail, one might have the following
     # config files:
     #
     #   /home/user1/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user1@mydomain.com with password mysecret is user1 here
     #     poll pop.myotherdomain.com proto pop3
     #       user user1@myotherdomain.com with password mysecrettoo is user1 here
     #
     #   /home/user2/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user2@mydomain.com with password yoursecret is user2 here
     #
     # This program would be run as follows:
     #
     #   fetchmail-config /home .fetchmailrc
     #
     # To use it for fetching of mail from a main source and a fallback source on
     # a redundant system, one might have the following config files:
     #
     #   /home/user1/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user1@mydomain.com with password mysecret is user1 here
     #     poll pop.myotherdomain.com proto pop3
     #       user user1@myotherdomain.com with password mysecrettoo is user1 here
     #     poll thebigguy proto pop3
     #       user user1 with password redundoh is user1 here
     #
     #   /home/user2/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user2@mydomain.com with password yoursecret is user2 here
     #     poll thebigguy proto pop3
     #       user user2 with password redundy is user2 here
     #
     # This program would be run as follows:
     #
     #   fetchmail-config /home .fetchmailrc --primary=thebigguy
     #     --secondary=pop.mydomain.com,pop.myotherdomain.com
     #
     # To use it for fetching of mail on the main system with a fallback to the
     # redundant source (e.g. to pick up the mail the system missed while down),
     # one might have the following config files:
     #
     #   /home/user1/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user1@mydomain.com with password mysecret is user1 here
     #     poll pop.myotherdomain.com proto pop3
     #       user user1@myotherdomain.com with password mysecrettoo is user1 here
     #     poll elredundoh proto pop3
     #       user user1 with password redundoh is user1 here
     #
     #   /home/user2/.fetchmailrc
     #     poll pop.mydomain.com proto pop3
     #       user user2@mydomain.com with password yoursecret is user2 here
     #     poll elredundoh proto pop3
     #       user user2 with password redundy is user2 here
     #
     # This program would be run as follows:
     #
     #   fetchmail-config /home .fetchmailrc
     #     --primary=pop.mydomain.com,pop.myotherdomain.com
     #     --secondary=elredundoh
     #
     # Note that this program is not as robust as fetchmail when it comes to
     # processing the config file.  It expects the config file lines to look as
     # shown above, viz., the first line begins with "poll", followed by the
     # system to poll, followed by the protocol to use.  Subsequent lines contain
     # the user information, etc., and are copied as is, up to the next line
     # beginning with "poll".
     #
     #
     # Returns
     # -------
     #
     # Fatal errors will cause this script to bail out via the Perl die function.
     # A message (hopefully explanatory) will be generated that describes the error.
     # The return code will be set to one of the allowable values for errno (see
     # the operating system error list in sys/errno.h).
     #
     # Successful completion will see the generated fetchmail config file written
     # to stdout, presumably from whence it may be piped directly to fetchmail.
     #
     use FileHandle;
     use File::Find;
     use Getopt::Long;
     use Net::Ping;
     use Sys::Hostname;
     #
     # Work areas.
     #
     my %Servers = ();
     my ($PollReq, $PollServ, $PollProto);
     my ($PriServ, $PriPing);
     my $PriAlive = 1;
     my ($ServLocal, $ServAddr);
     my (@ServList, $ServName);
     #
     # Config information.
     #
     my $HomeDir = "";                       # Home directory
     my $HomeDirLen = 0;
     my $ConfigFile = "";                    # Config file name
     my $Primary = "";                       # List of primary servers
     my $Secondary = "";                     # List of secondary servers
     #
     # Debugging information.
     #
     $Debugging = 0;                         # Set on for debugging
     #
     # Process the command line options.  If this fails, give a summary of the
     # program's usage.
     #
     ProcessOptions() || UsageInfo();
     #
     # Get the home directory and config file, if any.
     #
     $HomeDir = shift;
     $ConfigFile = shift;
     die "Home directory and/or configuration file name not specified"
       if (!defined($HomeDir) || ($HomeDir eq "")
         || !defined($ConfigFile) || ($ConfigFile eq ""));
     $HomeDir =~ s/\/$//; $HomeDirLen = length($HomeDir) + 1;
     #
     # See if the user is doing the primary/secondary thing.
     #
     die "Primary and secondary servers not both specified"
       if ((($Primary ne "") && ($Secondary eq ""))
         || (($Primary eq "") && ($Secondary ne "")));
     #
     # Check if the primary servers are alive.
     #
     if ($Primary ne "")
       {
       print("Primary servers are ".$Primary."\n" .
         "Secondary servers are ".$Secondary."\n") if ($Debugging);
     $PriPing = Net::Ping->new(icmp, 30);
     ($ServLocal) = (gethostbyname(hostname()))[4];
     $ServLocal = join(".", unpack("C4", $ServLocal));
     foreach $PriServ (split(/,/, $Primary))
       {
       #
       # See if this primary server is the one we're running on.  If so, force
       # the switch to the secondary.
       #
       ($ServAddr) = (gethostbyname($PriServ))[4];
       $ServAddr = join(".", unpack("C4", $ServAddr));
      if ($ServAddr eq $ServLocal)
        {
        print("Primary server ".$PriServ." is the local server\n")
          if ($Debugging);
        $PriAlive = 0;
        }
      #
      # Otherwise, see if we can ping the primary.
      #
      elsif (!$PriPing->ping($PriServ))
        {
        print("Primary server ".$PriServ." at ".$ServAddr." " .
          "appears to be dead\n") if ($Debugging);
        $PriAlive = 0;
        }
      }
     $PriPing->close();
     #
     # If the primary server is up.
     #
     if ($PriAlive) { @ServList = split(/,/, $Secondary); }
     else { @ServList = split(/,/, $Primary); }
     if ($Debugging)
       {
       print("Omitting servers\n");
       foreach $ServName (@ServList) { print("  ".$ServName."\n"); }
       }
     }

#
# Find all the files that match what we're looking for; #
print("Looking in ".$HomeDir." for ".$ConfigFile."\n") if ($Debugging);

     find(\&ReadConfig, $HomeDir);
     #
     # See if we found anything.  If not, its time to bail out.
     #
     die "No configuration files named ".$ConfigFile." found in ".$HomeDir."."
       if (!keys(%Servers));
     #
     # Emit all of the poll requests in a single config file.
     #
     foreach $PollReq (keys(%Servers))
       {
       ($PollServ, $PollProto) = split(/\|/, $PollReq);
       #
       # If we're checking primary/secondary do it now.
       #
       if ($Primary ne "")
         {
         $PriAlive = 0;
      foreach $ServName (@ServList)
        {
        if ($ServName eq $PollServ) { $PriAlive = 1; last; }
        }
      next if ($PriAlive);
      }

#
# Emit this poll request.
#
print("poll ".$PollServ." proto ".$PollProto."\n".$Servers{$PollReq}); }
#
# We're all done.
#
exit(0);

     ############################################################################
     sub ReadConfig
     #
     # $File::Find::dir                      Contains the current directory name.
     #
     # $_                                    Contains the current filename within
     #                                       the directory.
     #
     # $File::Find::name                     Contains the concatenation
     #                                       "$File::Find::dir/$_".
     #
     # $File::Find::prune                    Can be set to true in this routine
     #                                       in order to prune the tree; that is,
     #                                       find() will not descend into any
     #                                       directory when $File::Find::prune is
     #                                       set.
     #
     # This routine is called once for each file that is found in the directory
     # being searched.  It looks for config files that match the name given and
     # processes them, if found.  Only the first subdirectories under the top
     # level are searched.
     {
     my ($UpperDir, $ConfigHand, $Config, $ConfigLine);
     my $PollServer = "";
     #
     # See if we can prune this subdirectory (if its not the first level).
     #
     if (-d)
       {
       $UpperDir = substr($File::Find::name, $HomeDirLen);
     if ($UpperDir =~ /\//)
       {
       print("Pruning ".$File::Find::name."\n") if ($Debugging);
      $File::Find::prune = 1; return;
      }

}
#
# See if the file name matches what we're looking for. #
if ($_ ne $ConfigFile) { return; }
#
# Open up the config file to process it. #
print("Found ".$File::Find::name."\n") if ($Debugging);

     $ConfigHand = new FileHandle;
     if (!$ConfigHand->open("<".$File::Find::name))
       {
       print("Failed to open config file ".$File::Find::name."\n")
         if ($Debugging);
     return;
     }
     binmode($ConfigHand);
     #
     # Read the config file into a variable so that we can work on it.
     #
     if ($ConfigHand->read($Config, 100000) <= 0)
       {
       print("Couldn't read config file ".$File::Find::name."\n") if ($Debugging);
     return;
     }
     $ConfigHand->close();
     #
     # Process all of the poll requests.  They look like this:
     #
     #      poll pop.mydomain.com proto pop3
     #        user user1@mydomain.com with password mysecret is user1 here
     
     foreach $ConfigLine (split(/\n/, $Config))
       {
       next if ($ConfigLine =~ /^\s\/);
       #
       # Is this a new poll?
       #
       if ($ConfigLine =~ /^\spoll\s+([^\s]+)\s+proto[^\s]*\s+([^\s]+)/i)
         {
         print("  Poll = ".$1.", proto = ".$2."\n") if ($Debugging);
      $PollServer = $1."|".$2;
      }

#
# Otherwise, add this line to the existing poll. #
else

      {
      if (exists($Servers{$PollServer}))
        { $Servers{$PollServer} .= $ConfigLine."\n"; }
      else { $Servers{$PollServer} = $ConfigLine."\n"; }
      }

}

     return;
     }
     ############################################################################
     sub ProcessOptions
     #
     # Process command line options and set the appropriate variables.
     #
     {
     my ($Result);
     #
     # Get options using standard option processor.
     #
     $Result = &GetOptions(
         "Debug", \$Debugging,               # Debugging
         "Primary=s", \$Primary,             # Primary servers list
         "Secondary=s", \$Secondary);        # Secondary servers list
     #
     # Return success if options gotten OK.
     #
     return ($Result);
     }
     ############################################################################
     sub UsageInfo
     #
     # Routine to display program usage information.  Called whenever the command
     # line parameters don't make sense.
     #
     {
     #
     # Exit after giving help.
     #
     print <<USAGE ;