- Added protection for malformed From addresses. If the From does not at

least have an '@' in the address, then the signing domain is not extracted
   and the message will not be signed

CHANGES

@@ -1,8 +1,73 @@
+0.9.6 2018-03-13
+ - Fixed typo in package installation section of README
+ - Added more to README about first run with systemd
+ - Fixed typo in path for fallback location of the config file if one is not
+   provided
+ - Added protection for malformed From addresses.  If the From does not at
+   least have an '@' in the address, then the signing domain is not extracted
+   and the message will not be signed
 
-0.9.1 UNRELEASED
+0.9.5.1 2018-03-10
+ - Add conf file location to systemd unit file
+ - Fix setup.py install locations so they are installed correctly
+
+0.9.5 2018-03-10
+ - Beta 1 (updated Alpha -> Beta warning in README and trove classifiers)
+ - Added support for MacroList option
+ - Added support for MacroListVerify option
+ - Added example in README to show use of MacroList* to separate inbound and
+   outbound mail streams
+ - Added support for SyslogSuccess option (both signing and verifying)
+ - Rationalized logging to be much less verbose unless SyslogSuccess or
+   debugLevel are set - default is generally start/stop/errors only
+ - Fixed install_requires so either dnspython (preferred if neither is
+   installed) or PyDNS satisfies the install requirements
+ - Updated Authentication Results result comment not to mention key size for
+   ed25519 signatures, since it's irrelevant
+ - Enhanced signature verification logging to provide more useful information
+
+0.9.4 2018-03-09
+ - Create PID directory if it is missing
+ - Fix crash when verifying if domain for signing was not set
+ - Fix header folding to use \n only to align with milter protocol
+   requirements
+ - Added information about creating a dedicated user and PID file directory
+   creation to README
+ - Fixed a bug where dkim fail might be reported as pass when verifying
+   multiple signatures and a previous signature had passed
+ - Make RSA signatures in dkimpy-milter optional, so dkimpy-milter can be
+   added after an existing DKIM signing application to add an Ed25519
+   signature (Thanks to A. Schulze for the patch)
+ - Added support for AuthservID option
+ - Added support for InternalHosts option (ipaddress and either dns (dnspython)
+   or pydns (DNS) modules are now required)
+ - Added support for DiagnosticDirectory and updated dkimpy-milter specifics in
+   dkimpy-milter.conf.5
+
+0.9.3 2018-03-02
+ - Fixup csl dataset processing for single item lists
+ - file: dataset support
+ - Bump minimum authres version to 1.1.0 due to known issues with 1.0.2
+ - Ignore errors parsing broken authres header fields
+ - Fold added authres header fields
+ - Fix pidfile permissions
+ - Fix socket setup sequence so Unix sockets work
+
+0.9.2 2018-02-19
+ - Improved package requirements definition
+ - Added systemd unit file and (untested) sysv init file
+ - Added dkim-milter.8 (based on opendim.8)
+ - Implemented support for Canonicalization option
+ - Implemented support for SyslogFacility option
+ - Initial dataset support: csl
+ - Only sign if mail from from a domain in Domain and only if Mode is not
+   verfication only
+   
+0.9.1 2018-02-17
  - DKIM signing and verification using both RSA and Ed25519
  - The following configuration options are supported (same definition as
    OpenDKIM): Domain, KeyFile, KeyFileEd25519, Mode, PidFile, Selector,
    Socket, Syslog, UMask, and UserID (see dkimpy-milter.conf.5)
  - This is an Alpha grade release and while the implemented features work, it
    is nowhere near being a complete package
+

MANIFEST.in

@@ -1,5 +1,6 @@
 include etc/*
 include man/*
+include system/*
 include Authors.conf
 include TODO
 include README

README

@@ -13,6 +13,123 @@ default is a feature:
 
 python setup.py install --single-version-externally-managed --record=/dev/null
 
-WARNING: This is an alpha grade release to support interoperability testing with
-Ed25519 signatures and basic functionality.  It is known to be incomplete and
-not suitable for general use.
+For users of Debian Stable (Debian 9, Codename Squueze), all dependencies are
+available in either the main or backports repositories:
+
+[sudo] apt install python-milter python-nacl python-ipaddress python-dnspython
+[sudo] apt install -t stretch-backports python-authres python-dkim
+
+The preferred method of installation is from PyPi using pip (if distribution
+packages are not available):
+
+[sudo] pip install dkimpy_milter
+
+Using pip will cause required packages to be installed via easy_install if they
+have not been previously installed.
+
+The milter will work with either pydns (DNS) or dnspython (dns), preferring
+dnspython is both are available.  The dkimpy DKIM module also works with
+either.
+
+Both a systemd unit file and a sysv init file are provided.  Both make
+assumptions about defaults being used, e.g. if a non-standard pidfile name is
+used, they will need to be updated.  The sysv init file is Debian specific and
+untested, since the developers are not using sysv init.  Feedback/patches
+welcome.
+
+The dkimpy-milter drops priviledges after setup to the user/group specified in
+UserID.  During initial setup, this system user needs to be manually created.
+As an example, using the default dkimpy-user on Debian, the command would be:
+
+[sudo] adduser --system --no-create-home --quiet --disabled-password \
+               --disabled-login --shell /bin/false --group \
+               --home /var/run/dkimpy-milter dkimpy-milter
+
+Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
+missing, the milter will create it on startup.
+
+To start dkimpy-milter with systemd for the first time, you will need to take
+the following steps:
+
+[sudo] systemctl daemon-reload
+[sudo] systemctl enable dkimpy-milter
+[sudo] systemctl start dkimpy-milter
+[sudo] systemctl status dkimpy-milter (to verify it started correctly)
+
+As with all milters, dkimpy-milter needs to be integrated with your MTA of
+choice (Sendmail or Postfix).
+
+For Sendmail:
+
+Configuration is very similar to opendkim, but needs some adjustment for
+dkimpy-milter. Here's an example configuration line to include in your
+sendmail.mc:
+
+INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/var/run/dkimpy-milter/dkimpy-milter.sock')dnl
+
+Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
+and a restart of sendmail.  Note that S= needs to match the value of Socket in
+the dkimpy-milter configuration file.
+
+Milter support should be present by default in most versions of sendmail
+these days, but if not included in your Sendmail build, see:
+http://www.elandsys.com/resources/sendmail/milter.html
+
+For Postfix:
+
+Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
+README_FILES/MILTER_README).  Here's an example master.cf excerpt the talks to
+two dkimpy-milter instances, one configured for signing and one configured for
+verification:
+
+smtp       inet  n       -       -       -      -       smtpd
+    ...
+    -o smtpd_milters=inet:localhost:8892
+    ...
+
+submission inet  n       -       -       -      -       smtpd
+    ...
+    -o smtpd_milters=inet:localhost:8891
+    ...
+
+These need to match the Socket value for each dkimpy-milter instance.
+
+Care is required to segregate outbound mail to be signed and inbound mail to
+be verified.  The above example uses two instances of dkimpy-milter to do
+this.  There are many possible ways.  Here is another example using milter
+macros to keep the mail streams segregated:
+
+Postfix master.cf:
+
+smtp       inet  n       -       -       -       -       smtpd
+    ...
+    -o smtpd_milters=inet:localhost:8891
+    -o milter_macro_daemon_name=VERIFYING
+    ...
+
+submission inet n       -       -       -       -       smtpd
+    -o syslog_name=postfix/submission
+    -o smtpd_tls_security_level=encrypt
+    -o smtpd_sasl_auth_enable=yes
+    ...
+    -o milter_macro_daemon_name=ORIGINATING
+    -o smtpd_milters=inet:localhost:8891
+    ...
+
+Dkimpy-milter.conf:
+
+...
+Mode			sv
+MacroList		dameon_name|ORIGINATING
+MacroListVerify		daemon_name|VERIFYING
+...
+
+
+The python DKIM library, dkimpy, requires the entire message being signed or
+verified to be in memory, so dkimpy-milter does not write messages out to a temp
+file.  This may impact performance on low-memory systems.
+
+This is an beta grade release to support interoperability testing with Ed25519
+signatures sufficient functionality for basic use.  The documented
+functionality has been implemented and at least partially tested.  It is free
+of known major defects, but is not fully tested in a variety of environments.

TODO

@@ -15,36 +15,52 @@ UMask               implemented
 UserID              implemented     verified
 DKIM 'a' in AR      implemented     verified
 
+0.9.2 (Alpha)
+dkimpy-milter.service  implemented      verified
+sysv init              implemented      lightly tested
+remove PidFile on stop implemented      verified
+dkimpy-milter.8        provided         needs work
+Basic dataset (csl)    implemented      verified
+Sign based on Domain   implemented      verified
+Canonicalization       implemented      verified
+SyslogFacility         implemented      verified
+
+0.9.3 (Alpha)
+File dataset           implemented      verified
+
+0.9.4 (Alpha)
+AuthservID      implemented     verified
+DiagnosticDirectory     implemented     verified
+InternalHosts   implemented     verified
+
 0.9.5 (Beta)
-dkimpy-milter.8
-dkimpy-milter.service
-remove PidFile on stop
-AuthservID
-Canonicalization
-Diagnostics
-DiagnosticDirectory
-InternalHosts
-SyslogFacility
-SyslogSuccess
+MacroList	implemented	verified
+MacroListVerify	implemented	verified
+SyslogSuccess   implemented     verified
 
 1.0.0
 No additional features planned
 
+Plannedataset type support (if needed):
+db:/.db
+mdb:
+
 Considered for near-term feature release
 
 AlwaysAddARHeader
 ChangeRootDirectory
 ClockDrift (requires dkimpy change)
 DNSTimeout (requires dkmpy change)
-MacroList
 MilterDebug
 MinimumKeyBits
+OversignHeaders (may require dkimpy changes)
 PeerList
 SignatureAlgorithm
 
 Later
 
 BaseDirectory
+Diagnostics (requires dkimpy changes)
 DontSignMailTo
 ExemptDomains
 ExternalIgnoreList
@@ -68,7 +84,6 @@ On-InternalError
 On-KeyNotFound
 On-NoSignature
 On-SignatureError
-OversignHeaders
 RemoveARAll
 RemoveARFrom
 RemoveOldSignatures 

dkimpy_milter/__init__.py

@@ -25,216 +25,300 @@ import sys
 import syslog
 import Milter
 import dkim
-from dkim.dnsplug import get_txt
-from dkim.util import parse_tag_value
 import authres
 import os
 import tempfile
 import StringIO
 import re
-from Milter.config import MilterConfigParser
-from Milter.utils import iniplist,parse_addr,parseaddr
+from Milter.utils import parse_addr, parseaddr
 import dkimpy_milter.config as config
 from dkimpy_milter.util import drop_privileges
 from dkimpy_milter.util import setExceptHook
 from dkimpy_milter.util import write_pid
 from dkimpy_milter.util import read_keyfile
+from dkimpy_milter.util import own_socketfile
+from dkimpy_milter.util import fold
 
+__version__ = "0.9.6"
 FWS = re.compile(r'\r?\n[ \t]+')
 
+
 class dkimMilter(Milter.Base):
-  "Milter to check and sign DKIM.  Each connection gets its own instance."
+    "Milter to check and sign DKIM.  Each connection gets its own instance."
 
-  def __init__(self):
-    self.mailfrom = None
-    self.id = Milter.uniqueID()
-    # we don't want config used to change during a connection
-    self.conf = milterconfig
-    self.privatersa = privateRSA
-    self.privateed25519 = privateEd25519
-    self.fp = None
+    def __init__(self):
+        self.mailfrom = None
+        self.id = Milter.uniqueID()
+        # we don't want config used to change during a connection
+        self.conf = milterconfig
+        self.privatersa = privateRSA
+        self.privateed25519 = privateEd25519
+        self.fp = None
 
-  @Milter.noreply
-  def connect(self,hostname,unused,hostaddr):
-    self.internal_connection = False
-    self.hello_name = None
-    # sometimes people put extra space in sendmail config, so we strip
-    self.receiver = self.getsymval('j').strip()
-    if hostaddr and len(hostaddr) > 0:
-        ipaddr = hostaddr[0]
-        """if iniplist(ipaddr,self.conf.internal_connect): FIXME
-            self.internal_connection = True"""
-    else: ipaddr = ''
-    self.connectip = ipaddr
-    if self.internal_connection:
-        connecttype = 'INTERNAL'
-    else:
-        connecttype = 'EXTERNAL'
-    if milterconfig.get('Syslog'):
-        syslog.syslog("connect from {0} at {1} {2}".format(hostname,hostaddr,connecttype))
-    return Milter.CONTINUE
+    @Milter.noreply
+    def connect(self, hostname, unused, hostaddr):
+        self.internal_connection = False
+        self.external_connection = False
+        self.hello_name = None
+        # sometimes people put extra space in sendmail config, so we strip
+        self.receiver = self.getsymval('j').strip()
+        try:
+            self.AuthservID = milterconfig['AuthservID']
+        except:
+            self.AuthservID = self.receiver
+        if hostaddr and len(hostaddr) > 0:
+            ipaddr = hostaddr[0]
+            if milterconfig['IntHosts']:
+                if milterconfig['IntHosts'].match(ipaddr):
+                    self.internal_connection = True
+        else:
+            ipaddr = ''
+        self.connectip = ipaddr
+        if milterconfig.get('MacroList') and not self.internal_connection:
+            macrolist = milterconfig.get('MacroList')
+            for macro in macrolist:
+                macroname = macro.split('|')[0]
+                macroname = '{' + macroname + '}'
+                macroresult = self.getsymval(macroname)
+                if ((len(macro.split('|')) == 1 and macroresult) or macroresult
+                        in macro.split('|')[1:]):
+                    self.internal_connection = True
+        if milterconfig.get('MacroListVerify'):
+            macrolist = milterconfig.get('MacroListVerify')
+            for macro in macrolist:
+                macroname = macro.split('|')[0]
+                macroname = '{' + macroname + '}'
+                macroresult = self.getsymval(macroname)
+                if ((len(macro.split('|')) == 1 and macroresult) or macroresult
+                        in macro.split('|')[1:]):
+                    self.external_connection = True
+        if self.internal_connection:
+            connecttype = 'INTERNAL'
+        else:
+            connecttype = 'EXTERNAL'
+        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 1:
+            syslog.syslog("connect from {0} at {1} {2}"
+                          .format(hostname, hostaddr, connecttype))
+        return Milter.CONTINUE
 
-  # multiple messages can be received on a single connection
-  # envfrom (MAIL FROM in the SMTP protocol) seems to mark the start
-  # of each message.
-  @Milter.noreply
-  def envfrom(self,f,*str):
-    if milterconfig.get('Syslog'):  
-        syslog.syslog("mail from: {0} {1}".format(f,str))
-    self.fp = StringIO.StringIO()
-    self.mailfrom = f
-    t = parse_addr(f)
-    if len(t) == 2: t[1] = t[1].lower()
-    self.canon_from = '@'.join(t)
-    self.user = self.getsymval('{auth_authen}')
-    self.has_dkim = 0
-    self.author = None
-    self.arheaders = []
-    self.arresults = []
-    '''if self.user:
-      # Very simple SMTP AUTH policy by default:
-      #   any successful authentication is considered INTERNAL
-      self.internal_connection = True
-      auth_type = self.getsymval('{auth_type}')
-      ssl_bits =  self.getsymval('{cipher_bits}')
-      if milterconfig.get('Syslog'):
-          syslog.syslog(
-              "SMTP AUTH:",self.user,"sslbits =",ssl_bits, auth_type,
-              "ssf =",self.getsymval('{auth_ssf}'), "INTERNAL"
-      )
-      # Detailed authorization policy is configured in the access file below.
-      self.arresults.append(
-        authres.SMTPAUTHAuthenticationResult(result = 'pass',
-      	result_comment = auth_type+' sslbits='+ssl_bits, smtp_auth = self.user)
-      )'''
-    return Milter.CONTINUE
+    # multiple messages can be received on a single connection
+    # envfrom (MAIL FROM in the SMTP protocol) seems to mark the start
+    # of each message.
+    @Milter.noreply
+    def envfrom(self, f, *str):
+        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 2:
+            syslog.syslog("mail from: {0} {1}".format(f, str))
+        self.fp = StringIO.StringIO()
+        self.mailfrom = f
+        t = parse_addr(f)
+        if len(t) == 2:
+            t[1] = t[1].lower()
+        self.canon_from = '@'.join(t)
+        self.has_dkim = 0
+        self.author = None
+        self.arheaders = []
+        self.arresults = []
+        return Milter.CONTINUE
 
-  @Milter.noreply
-  def header(self,name,val):
-    lname = name.lower()
-    if lname == 'dkim-signature':
-        if milterconfig.get('Syslog'):
-          syslog.syslog("{0}: {1}".format(name,val))
-        self.has_dkim += 1
-    if lname == 'from':
-        fname,self.author = parseaddr(val)
-        if milterconfig.get('Syslog'):
-            syslog.syslog("{0}: {1}".format(name,val))
-    elif lname == 'authentication-results':
-        self.arheaders.append(val)
-    if self.fp:
-        self.fp.write("%s: %s\n" % (name,val))
-    return Milter.CONTINUE
+    @Milter.noreply
+    def header(self, name, val):
+        lname = name.lower()
+        if lname == 'dkim-signature':
+            if (milterconfig.get('Syslog') and
+                    milterconfig.get('debugLevel') >= 1):
+                syslog.syslog("{0}: {1}".format(name, val))
+            self.has_dkim += 1
+        if lname == 'from':
+            fname, self.author = parseaddr(val)
+            try:
+                self.fdomain = self.author.split('@')[1]
+            except IndexError as er:
+                self.fdomain = ''  # self.author was not a proper email address
+            if (milterconfig.get('Syslog') and
+                    milterconfig.get('debugLevel') >= 1):
+                syslog.syslog("{0}: {1}".format(name, val))
+        elif lname == 'authentication-results':
+            self.arheaders.append(val)
+        if self.fp:
+            self.fp.write("%s: %s\n" % (name, val))
+        return Milter.CONTINUE
 
-  @Milter.noreply
-  def eoh(self):
-    if self.fp:
-      self.fp.write("\n")                         # terminate headers
-    self.bodysize = 0
-    return Milter.CONTINUE
+    @Milter.noreply
+    def eoh(self):
+        if self.fp:
+            self.fp.write("\n")   # terminate headers
+        self.bodysize = 0
+        return Milter.CONTINUE
 
-  @Milter.noreply
-  def body(self,chunk):         # copy body to temp file
-    if self.fp:
-      self.fp.write(chunk)      # IOError causes TEMPFAIL in milter
-      self.bodysize += len(chunk)
-    return Milter.CONTINUE
+    @Milter.noreply
+    def body(self, chunk):        # copy body to temp file
+        if self.fp:
+            self.fp.write(chunk)  # IOError causes TEMPFAIL in milter
+            self.bodysize += len(chunk)
+        return Milter.CONTINUE
 
-  def eom(self):
-    if not self.fp:
-      return Milter.ACCEPT      # no message collected - so no eom processing
-    # Remove existing Authentication-Results headers for our authserv_id
-    for i,val in enumerate(self.arheaders,1):
-      # FIXME: don't delete A-R headers from trusted MTAs
-      ar = authres.AuthenticationResultsHeader.parse_value(FWS.sub('',val))
-      if ar.authserv_id == self.receiver:
-        self.chgheader('authentication-results',i,'')
-        if milterconfig.get('Syslog'):
-            syslog.syslog('REMOVE: {0}'.format(val))
-    # Check or sign DKIM
-    self.fp.seek(0)
-    if self.internal_connection or milterconfig.get('Mode') == 's' or milterconfig.get('Mode') == 'sv':
-      txt = self.fp.read()
-      self.sign_dkim(txt)
-      result = None
-    if self.has_dkim and (milterconfig.get('Mode') == 'v' or milterconfig.get('Mode') == 'sv'):
-      txt = self.fp.read()
-      self.check_dkim(txt)
-    else:
-      result = 'none'
-    if self.arresults:
-        h = authres.AuthenticationResultsHeader(authserv_id = self.receiver, 
-            results=self.arresults)
-        if milterconfig.get('Syslog'):
-            syslog.syslog(str(h))
-        name,val = str(h).split(': ',1)
-        self.addheader(name,val,0)
-    return Milter.CONTINUE
+    def eom(self):
+        if not self.fp:
+            return Milter.ACCEPT  # no message collected - so no eom processing
+        # Remove existing Authentication-Results headers for our authserv_id
+        for i, val in enumerate(self.arheaders, 1):
+            # FIXME: don't delete A-R headers from trusted MTAs
+            try:
+                ar = (authres.AuthenticationResultsHeader
+                      .parse_value(FWS.sub('', val)))
+                if ar.authserv_id == self.AuthservID:
+                    self.chgheader('authentication-results', i, '')
+                    if (milterconfig.get('Syslog') and
+                            milterconfig.get('debugLevel') >= 1):
+                        syslog.syslog('REMOVE: {0}'.format(val))
+            except:
+                # Don't error out on unparseable AR header fiels
+                pass
+        # Check or sign DKIM
+        self.fp.seek(0)
+        if milterconfig.get('Domain'):
+            domain = milterconfig.get('Domain')
+        else:
+            domain = ''
+        if ((self.fdomain in domain) and not milterconfig.get('Mode') == 'v'
+                and not self.external_connection):
+            txt = self.fp.read()
+            self.sign_dkim(txt)
+        if ((self.has_dkim) and (not self.internal_connection) and
+            (milterconfig.get('Mode') == 'v' or
+             milterconfig.get('Mode') == 'sv')):
+            txt = self.fp.read()
+            self.check_dkim(txt)
+        if self.arresults:
+            h = authres.AuthenticationResultsHeader(authserv_id=
+                                                    self.AuthservID,
+                                                    results=self.arresults)
+            h = fold(str(h))
+            if (milterconfig.get('Syslog') and
+                    milterconfig.get('debugLevel') >= 2):
+                syslog.syslog(str(h))
+            name, val = str(h).split(': ', 1)
+            self.addheader(name, val, 0)
+        return Milter.CONTINUE
 
-  def sign_dkim(self,txt):
-      conf = self.conf
-      try:
-        d = dkim.DKIM(txt)
-        h = d.sign(milterconfig.get('Selector'),milterconfig.get('Domain'), privateRSA,
-                canonicalize=('relaxed','simple'))
-        name,val = h.split(': ',1)
-        self.addheader(name,val.strip().replace('\r\n','\n'),0)
-        if privateEd25519:
-            d = dkim.DKIM(txt)
-            h = d.sign(milterconfig.get('SelectorEd25519'),milterconfig.get('Domain'), privateEd25519,
-                    canonicalize=('relaxed','simple'), signature_algorithm='ed25519-sha256')
-            name,val = h.split(': ',1)
-            self.addheader(name,val.strip().replace('\r\n','\n'),0)
-      except dkim.DKIMException as x:
-          if milterconfig.get('Syslog'):
-              syslog.syslog('DKIM: {0}'.format(x))
-      except Exception as x:
-          if milterconfig.get('Syslog'):
-              syslog.syslog("sign_dkim: {0}".format(x))
-          raise
-      
-  def check_dkim(self,txt):
-      res = False
-      conf = self.conf
-      for y in range(self.has_dkim): # Verify _ALL_ the signatures
-          d = dkim.DKIM(txt)
-          try:
-            res = d.verify(idx=y)
-            if res:
-              self.dkim_comment = 'Good {0} bit {1} signature.'.format(d.keysize, d.signature_fields.get(b'a'))
-            else:
-              self.dkim_comment = 'Bad {0} bit {1} signature.'.format(d.keysize, d.signature_fields.get(b'a'))
-          except dkim.DKIMException as x:
-            self.dkim_comment = str(x)
+    def sign_dkim(self, txt):
+        canon = milterconfig.get('Canonicalization')
+        canonicalize = []
+        if len(canon.split('/')) == 2:
+            canonicalize.append(canon.split('/')[0])
+            canonicalize.append(canon.split('/')[1])
+        else:
+            canonicalize.append(canon)
+            canonicalize.append(canon)
+            if (milterconfig.get('Syslog') and
+                    milterconfig.get('debugLevel') >= 1):
+                syslog.syslog('canonicalize: {0}'.format(canonicalize))
+        try:
+            if privateRSA:
+                d = dkim.DKIM(txt)
+                h = d.sign(milterconfig.get('Selector'), self.fdomain,
+                           privateRSA, canonicalize=(canonicalize[0],
+                                                     canonicalize[1]))
+                name, val = h.split(': ', 1)
+                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
+                if (milterconfig.get('Syslog') and
+                    (milterconfig.get('SyslogSuccess')
+                     or milterconfig.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b's'),
+                                                  d.domain))
+            if privateEd25519:
+                d = dkim.DKIM(txt)
+                h = d.sign(milterconfig.get('SelectorEd25519'), self.fdomain,
+                           privateEd25519, canonicalize=(canonicalize[0],
+                                                         canonicalize[1]),
+                           signature_algorithm='ed25519-sha256')
+                name, val = h.split(': ', 1)
+                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
+                if (milterconfig.get('Syslog') and
+                    (milterconfig.get('SyslogSuccess')
+                     or milterconfig.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b's'),
+                                                  d.domain))
+        except dkim.DKIMException as x:
             if milterconfig.get('Syslog'):
                 syslog.syslog('DKIM: {0}'.format(x))
-          except Exception as x:
-            self.dkim_comment = str(x)
+        except Exception as x:
             if milterconfig.get('Syslog'):
-                syslog.syslog("check_dkim: {0}".format(x))
-          self.header_i = d.signature_fields.get(b'i')
-          self.header_d = d.signature_fields.get(b'd')
-          self.header_a = d.signature_fields.get(b'a')
-          if res:
-              if milterconfig.get('Syslog'):
-                  syslog.syslog('DKIM: Pass ({0})'.format(d.domain))
-              self.dkim_domain = d.domain
-          else:
-            fd,fname = tempfile.mkstemp(".dkim")
-            with os.fdopen(fd,"w+b") as fp:
-                fp.write(txt)
-            if milterconfig.get('Syslog'):
-                syslog.syslog('DKIM: Fail (saved as {0})'.format(fname))
-          if res:
-            result = 'pass'
-          else:
-            result = 'fail'
-          self.arresults.append(
-            authres.DKIMAuthenticationResult(result=result,
-              header_i = self.header_i, header_d = self.header_d, header_a = self.header_a,
-              result_comment = self.dkim_comment)
-          )
-      return
+                syslog.syslog("sign_dkim: {0}".format(x))
+            raise
+
+    def check_dkim(self, txt):
+        res = False
+        for y in range(self.has_dkim):  # Verify _ALL_ the signatures
+            d = dkim.DKIM(txt)
+            try:
+                res = d.verify(idx=y)
+                if res:
+                    if d.signature_fields.get(b'a') == 'ed25519-sha256':
+                        self.dkim_comment = ('Good {0} signature'
+                                             .format(d.signature_fields
+                                                     .get(b'a')))
+                    else:
+                        self.dkim_comment = ('Good {0} bit {1} signature'
+                                             .format(d.keysize,
+                                                     d.signature_fields
+                                                     .get(b'a')))
+                else:
+                    self.dkim_comment = ('Bad {0} bit {1} signature.'
+                                         .format(d.keysize,
+                                                 d.signature_fields.get(b'a')))
+            except dkim.DKIMException as x:
+                self.dkim_comment = str(x)
+                if milterconfig.get('Syslog'):
+                    syslog.syslog('DKIM: {0}'.format(x))
+            except Exception as x:
+                self.dkim_comment = str(x)
+                if milterconfig.get('Syslog'):
+                    syslog.syslog("check_dkim: {0}".format(x))
+            self.header_i = d.signature_fields.get(b'i')
+            self.header_d = d.signature_fields.get(b'd')
+            self.header_a = d.signature_fields.get(b'a')
+            if res:
+                if (milterconfig.get('Syslog') and
+                        (milterconfig.get('SyslogSuccess') or
+                         milterconfig.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature verified (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b's'),
+                                                  d.domain))
+                self.dkim_domain = d.domain
+            else:
+                if milterconfig.get('DiagnosticDirectory'):
+                    fd, fname = tempfile.mkstemp(".dkim")
+                    with os.fdopen(fd, "w+b") as fp:
+                        fp.write(txt)
+                    if milterconfig.get('Syslog'):
+                        syslog.syslog('DKIM: Fail (saved as {0})'
+                                      .format(fname))
+                else:
+                    syslog.syslog('DKIM: Fail ({0})'.format(d.domain))
+            if res:
+                result = 'pass'
+            else:
+                result = 'fail'
+            res = False
+            self.arresults.append(
+                authres.DKIMAuthenticationResult(result=result,
+                                                 header_i=self.header_i,
+                                                 header_d=self.header_d,
+                                                 header_a=self.header_a,
+                                                 result_comment=
+                                                 self.dkim_comment)
+            )
+        return
+
 
 def main():
     # Ugh, but there's no easy way around this.
@@ -243,30 +327,34 @@ def main():
     global privateEd25519
     privateRSA = False
     privateEd25519 = False
-    configFile = '/etc/dkimpy-milter.conf'
+    configFile = '/usr/local/etc/dkimpy-milter.conf'
     if len(sys.argv) > 1:
-        if sys.argv[1] in ( '-?', '--help', '-h' ):
+        if sys.argv[1] in ('-?', '--help', '-h'):
             print('usage: dkimpy-milter [<configfilename>]')
             sys.exit(1)
         configFile = sys.argv[1]
-    milterconfig = config._processConfigFile(filename = configFile)
+    milterconfig = config._processConfigFile(filename=configFile)
     if milterconfig.get('Syslog'):
-        syslog.openlog(os.path.basename(sys.argv[0]), syslog.LOG_PID, syslog.LOG_MAIL)
+        facility = eval("syslog.LOG_{0}"
+                        .format(milterconfig.get('SyslogFacility').upper()))
+        syslog.openlog(os.path.basename(sys.argv[0]), syslog.LOG_PID, facility)
         setExceptHook()
-    write_pid(milterconfig)
+    pid = write_pid(milterconfig)
     if milterconfig.get('KeyFile'):
         privateRSA = read_keyfile(milterconfig, 'RSA')
     if milterconfig.get('KeyFileEd25519'):
         privateEd25519 = read_keyfile(milterconfig, 'Ed25519')
-    drop_privileges(milterconfig)
-    if milterconfig.get('Syslog'):
-        syslog.syslog('dkimpy-milter started.  user: {0}'.format(milterconfig.get('UserID')))
     Milter.factory = dkimMilter
     Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
     miltername = 'dkimpy-filter'
     socketname = milterconfig.get('Socket')
+    if milterconfig.get('Syslog'):
+        syslog.syslog('dkimpy-milter started:{0} user:{1}'
+                      .format(pid, milterconfig.get('UserID')))
     sys.stdout.flush()
-    Milter.runmilter(miltername,socketname,240)
+    Milter.runmilter(miltername, socketname, 240)
+    own_socketfile(milterconfig)
+    drop_privileges(milterconfig)
 
 if __name__ == "__main__":
     main()

dkimpy_milter/config.py

@@ -27,39 +27,207 @@
 import syslog
 import os
 import sys
-import re
-import urllib
 import stat
 import dkim
-
+import socket
+import ipaddress
+from dnsplug import Session
 
 #  default values
 defaultConfigData = {
-        'Syslog' : 'yes',
-        'SyslogFacility' : 'mail',
-        'UMask' : 007,
-        'Mode' : 'sv',
-        'Socket' : 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
-        'PidFile'  : '/var/run/dkimpy-milter/dkimpy-milter.pid',
-        'UserID' : 'dkimpy-milter',
-        'Canonicalization' : 'simple'
-        }
+    'Syslog': 'yes',
+    'SyslogFacility': 'mail',
+    'UMask': 007,
+    'Mode': 'sv',
+    'Socket': 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
+    'PidFile': '/var/run/dkimpy-milter/dkimpy-milter.pid',
+    'UserID': 'dkimpy-milter',
+    'Canonicalization': 'relaxed/simple',
+    'InternalHosts': '127.0.0.1',
+    'IntHosts': False,
+    'DiagnosticDirectory': '',
+    'MacroList': '',
+    'MacroListVerify': '',
+    'debugLevel': 0  # Undocumented config item for developer use
+    }
 
 
-#################################
 class ConfigException(Exception):
     '''Exception raised when there's a configuration file error.'''
     pass
 
-####################################################################
-def _processConfigFile(filename = None, configdata = None, useSyslog = 1,
-        useStderr = 0):
+
+class HostsDataset(object):
+    '''Hold a group of host related dataset objects'''
+
+    def __init__(self, dataset):
+        self.dataset = []
+        # Self.dataset will end up being a list of DataSetItem(s).
+        for item in dataset:
+            item = item.rstrip(']')
+            item = item.lstrip('[')
+            self.dataset.append(self.DatasetItem(item))
+
+    class DatasetItem(object):
+        '''Individual dataset item'''
+
+        def __init__(self, item):
+            self.item = item
+            self.isipv4 = False
+            self.isipv4cidr = False
+            self.isipv6 = False
+            self.isipv6cidr = False
+            self.ishostname = False
+            self.isdomain = False
+            self.negative = False
+            if self.item[0] == '!':
+                self.item = item[1:]
+                self.negative = True
+            try:
+                self.item = ipaddress.ip_address(unicode(self.item, "utf-8"))
+                if isinstance(self.item, ipaddress.IPv4Address):
+                    self.isipv4 = True
+                elif isinstance(self.item, ipaddress.IPv6Address):
+                    self.isipv6 = True
+            except ValueError as e:
+                try:
+                    self.item = ipaddress.ip_network(unicode
+                                                     (self.item, "utf-8"),
+                                                     strict=False)
+                    if isinstance(self.item, ipaddress.IPv4Network):
+                        self.isipv4cidr = True
+                    elif isinstance(self.item, ipaddress.IPv6Network):
+                        self.isipv6cidr = True
+                except ValueError as e2:
+                    if self.item[0] == '.' and len(self.item.split('.')) > 2:
+                        self.isdomain = True
+                    elif len(self.item.split('.')) > 1:  # It has a '.' in it
+                        self.ishostname = True
+                    else:
+                        raise ConfigException('Unknown dataset item: {0}'
+                                              .format(item))
+
+    def match(self, connectip):
+        '''Check if the connect IP is part of the dataset'''
+        source = ipaddress.ip_address(unicode(connectip, "utf-8"))
+        for item in self.dataset:
+            if item.isdomain or item.ishostname:
+                result = self.matchname(source)   # Match host/domains first
+                if result:
+                    return(result)
+            elif item.isipv4 or item.isipv4cidr:  # Then IPv4/6 addresses or
+                if isinstance(source, ipaddress.IPv4Address):  # networks
+                    return(self.match4(source))   # depending on the item type
+            elif item.isipv6 or item.isipv6cidr:  # and connect type
+                if isinstance(source, ipaddress.IPv6Address):
+                    return(self.match6(source))
+
+    def matchname(self, source):
+        '''Does source IP address relate to a domain/hostname in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchdomain = False
+        negativedomain = False
+        ptrlist = self.getptr(source)
+        for item in self.dataset:
+            if item.isdomain:
+                for ptr in ptrlist:
+                    # Strip the leading '.' off the domain name for exact match
+                    if item.item[1:] == ptr[-len(item.item)+1:]:
+                        matchdomain = True
+                        negativedomain = item.negative
+            elif item.ishostname:
+                for ptr in ptrlist:
+                    if item.item == ptr:
+                        matchone = True
+                        negativeone = item.negative
+        if matchdomain and not negativedomain:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+    def getptr(self, source):
+        '''Get validated PTR name of IP address'''
+        results = []
+        s = Session()
+        ptrnames = s.dns(source.reverse_pointer, 'PTR')
+        for name in ptrnames:
+            if isinstance(source, ipaddress.IPv4Address):
+                ips = s.dns(name, 'A')
+                for ip in ips:
+                    ip = ipaddress.IPv4Address(unicode(ip, 'UTF-8'))
+                    if ip == source:
+                        results.append(name)
+            if isinstance(source, ipaddress.IPv6Address):
+                ips = s.dns(name, 'AAAA')
+                for ip in ips:
+                    ip = ipaddress.IPv6Address(unicode(ip, 'UTF-8'))
+                if ip == source:
+                    results.append(name)
+        return results
+
+    def match4(self, source):
+        '''Is the source IP related to a IPv4 address/network in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchcidr = False
+        negativecidr = False
+        for item in self.dataset:
+            if item.isipv4:
+                if source == item.item:
+                    matchone = True
+                    negativeone = item.negative
+            elif item.isipv4cidr:
+                if source in item.item:
+                    matchcidr = True
+                    negativecidr = item.negative
+        if matchcidr and not negativecidr:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+    def match6(self, source):
+        '''Is the source IP realted to a IPv6 address/network in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchcidr = False
+        negativecidr = False
+        for item in self.dataset:
+            if item.isipv6:
+                if source == item.item:
+                    matchone = True
+                    negativeone = item.negative
+            elif item.isipv6cidr:
+                if source in item.item:
+                    matchcidr = True
+                    negativecidr = item.negative
+        if matchcidr and not negativecidr:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+
+def _processConfigFile(filename=None, configdata=None, useSyslog=1,
+                       useStderr=0):
     '''Load the specified config file, exit and log errors if it fails,
     otherwise return a config dictionary.'''
 
     import config
-    if configdata == None: configdata = config.defaultConfigData
-    if filename != None:
+    if configdata is None:
+        configdata = config.defaultConfigData
+    if filename is not None:
         try:
             _readConfigFile(filename, configdata)
         except Exception, e:
@@ -71,7 +239,7 @@ def _processConfigFile(filename = None, configdata = None, useSyslog = 1,
             sys.exit(1)
     return(configdata)
 
-####################
+
 def _find_boolean(item):
     if type(item) == int:
         item = str(item)
@@ -84,84 +252,168 @@ def _find_boolean(item):
     return item
 
 
-###############################################################
-commentRx = re.compile(r'^(.*)#.*$')
-def _readConfigFile(path, configData = None, configGlobal = {}):
+def _make_authserv_id(as_id):
+    """Determine AuthservID if needed"""
+    if as_id == 'HOSTNAME':
+        as_id = socket.gethostname()
+    return as_id
+
+
+def _dataset_to_list(dataset):
+    """Convert a dataset (as defined in dkimpymilter.8) and return a python
+       list of values."""
+    if not isinstance(dataset, str):
+        # If it was a csl with more than one value, it's already a list, we
+        # only need to remove the name from the first value.
+        if dataset[0][:4] == 'csl:':
+            dataset[0] = dataset[0][4:]
+        for item in dataset:
+            dataset[dataset.index(item)] = item.strip().strip(',')
+        return dataset
+    elif isinstance(dataset, str):
+        if dataset[0] == '/' or dataset[:5] == 'file:':
+            # This is a flat file dataset
+            ds = []
+            if dataset[0] == '/':
+                dsname = dataset
+            if dataset[:5] == 'file:':
+                dsname = dataset[5:]
+            dsf = open(dsname, 'r')
+            for line in dsf.readlines():
+                if line[0] != '#':
+                    if len(line.split(':')) == 1:
+                        ds.append(line.strip())
+                    else:
+                        for element in line.split(':'):
+                            ds.append(element.strip().strip(':'))
+            dsf.close()
+            return ds
+        # If it's a str and csl, it has one value and we return a list
+        if dataset[:4] == 'csl:':
+            return [dataset[4:].strip().strip(',')]
+        else:
+            return [dataset.strip().strip(',')]
+        if dataset[-3:] == '.db' or dataset[:3] == 'db:':
+            #  This is a Sleepycat (Oracle) DB  dataset
+            import whichdb  # Will need rewriting someday for python3
+            if dataset[-3:] == '.db':
+                dbname = dataset
+            elif dataset[:3] == 'db:':
+                dbname = dataset[3:]
+            else:
+                raise dkim.ParameterError('Unimplmented dataset type: {0}'
+                                          .format(type(dataset)))
+            if whichdb.whichdb(dbname) != 'dbhash':
+                raise dkim.ParameterError('Unimplmented dataset type: {0}'
+                                          .format(type(dataset)))
+            #TODO replace this with code to use db maps
+            raise dkim.ParameterError('Unsupported dataset db datase: {0}'
+                                      .format(type(dataset)))
+
+    raise dkim.ParameterError('Unimplmented dataset type: {0}'
+                              .format(type(dataset)))
+
+
+def _readConfigFile(path, configData=None, configGlobal={}):
     '''Reads a configuration file from the specified path, merging it
     with the configuration data specified in configData.  Returns a
     dictionary of name/value pairs based on configData and the values
     read from path.'''
 
     debugLevel = configGlobal.get('debugLevel', 0)
-    if debugLevel >= 5: syslog.syslog('readConfigFile: Loading "%s"' % path)
-    if configData == None: configData = {}
+    if debugLevel >= 5:
+        syslog.syslog('readConfigFile: Loading "%s"' % path)
+    if configData is None:
+        configData = {}
     nameConversion = {
-        'AuthservID' : 'str',
-        'Syslog' : 'bool',
-        'SyslogFacility' : 'str',
-        'SyslogSuccess' : 'bool',
-        'UMask' : 'int',
-        'Mode' : 'str',
-        'Socket' : 'str',
-        'PidFile'  : 'str',
-        'UserID' : 'str',
-        'Domain' : 'str',
-        'KeyFile' : 'str',
-        'KeyFileEd25519' : 'str',
-        'Selector' : 'str',
+        'AuthservID': 'str',
+        'Syslog': 'bool',
+        'SyslogFacility': 'str',
+        'SyslogSuccess': 'bool',
+        'UMask': 'int',
+        'Mode': 'str',
+        'Socket': 'str',
+        'PidFile': 'str',
+        'UserID': 'str',
+        'Domain': 'dataset',
+        'KeyFile': 'str',
+        'KeyFileEd25519': 'str',
+        'Selector': 'str',
         'SelectorEd25519': 'str',
-        'Canonicalization' : 'str',
-        'CanonicalizationEd25519' : 'str'
-            }
+        'Canonicalization': 'str',
+        'InternalHosts': 'dataset',
+        'IntHosts': 'bool',
+        'DiagnosticDirectory': 'str',
+        'MacroList': 'dataset',
+        'MacroListVerify': 'dataset',
+        'debugLevel': 'int'
+        }
 
     #  check to see if it's a file
     try:
         mode = os.stat(path)[0]
     except OSError, e:
-        syslog.syslog(syslog.LOG_ERR,'ERROR stating "%s": %s' % ( path, e.strerror ))
+        syslog.syslog(syslog.LOG_ERR, 'ERROR stating "%s": %s'
+                      % (path, e.strerror))
         return(configData)
     if not stat.S_ISREG(mode):
-        syslog.syslog(syslog.LOG_ERR,'ERROR: is not a file: "%s", mode=%s' % ( path, oct(mode) ))
+        syslog.syslog(syslog.LOG_ERR, 'ERROR: is not a file: "%s", mode=%s'
+                      % (path, oct(mode)))
         return(configData)
 
     #  load file
     fp = open(path, 'r')
     while 1:
         line = fp.readline()
-        if not line: break
+        if not line:
+            break
 
         #  parse line
         line = line.split('#', 1)[0].strip()
-        if not line: continue
+        if not line:
+            continue
         data = line.split()
         if len(data) != 2:
             if len(data) == 1:
                 if debugLevel >= 1:
-                    syslog.syslog('Configuration item "%s" not defined in file "%s"'
-                        % ( line, path ))
-            else:
-                syslog.syslog('ERROR parsing line "%s" from file "%s"'
-                    % ( line, path ))
-            continue
-        name, value = data
+                    syslog.syslog('Config item "%s" not defined in file "%s"'
+                                  % (line, path))
+        if len(data) == 1:
+            name = data
+            value = ''
+        if len(data) == 2:
+            name, value = data
+        if len(data) >= 3:
+            name = data[0]
+            value = data[1:]
 
         #  check validity of name
         conversion = nameConversion.get(name)
-        if conversion == None:
-            syslog.syslog('ERROR: Unknown name "%s" in file "%s"' % ( name, path ))
+        if conversion is None:
+            syslog.syslog('ERROR: Unknown name "%s" in file "%s"'
+                          % (name, path))
             continue
 
-        if debugLevel >= 5: syslog.syslog('readConfigFile: Found entry "%s=%s"'
-                % ( name, value ))
+        if debugLevel >= 5:
+            syslog.syslog('readConfigFile: Found entry "%s=%s"'
+                          % (name, value))
         if conversion == 'bool':
             configData[name] = _find_boolean(value)
         elif conversion == 'str':
             configData[name] = str(value)
         elif conversion == 'int':
             configData[name] = int(value)
+        elif conversion == 'dataset':
+            configData[name] = _dataset_to_list(value)
         else:
-            syslog.syslog(str('name: ' + name + ' value: ' + value + ' conversion: ' + conversion))
+            syslog.syslog(str('name: ' + name + ' value: ' + value +
+                              ' conversion: ' + conversion))
             configData[name] = conversion(value)
     fp.close()
+    try:
+        configData['AuthservID'] = _make_authserv_id(configData['AuthservID'])
+        configData['IntHosts'] = HostsDataset(configData['InternalHosts'])
+    except:
+        pass
 
     return(configData)

dkimpy_milter/dnsplug.py

@@ -0,0 +1,168 @@
+## @package dnsplug
+# Provide a higher level interface to pydns or dnspython (or other provider).
+# NOT RELEASED: this is a proposed API and implementation.
+# Goals - work with both pydns and dnspython (and possibly other libraries)
+# at a simplied level.
+# TODO:
+# 1. map exceptions to common dnsplug.DNSError exception (with
+#    original exception saved as a member).
+# 2. include dict based implementation (handy for test suites)
+# 3. move implementations to subpackages to enable autoselect on first call.
+
+## Maximum number of CNAME records to follow
+MAX_CNAME = 10
+
+## Lookup DNS records by label and RR type.
+# The response can include records of other types that the DNS
+# server thinks we might need.  FIXME: empty result
+# could mean NXDOMAIN or NOANSWER.
+# @param name the DNS label to lookup
+# @param qtype the name of the DNS RR type to lookup
+# @param tcpfallback if False, raise exception instead of TCP fallback
+# @return a list of ((name,type),data) tuples
+def DNSLookup(name, qtype, tcpfallback=True, timeout=30):
+    raise NotImplementedError('No supported dns library found')
+
+class Session(object):
+  """A Session object has a simple cache with no TTL that is valid
+   for a single "session", for example an SMTP conversation."""
+  def __init__(self):
+    self.cache = {}
+
+  ## Additional DNS RRs we can safely cache.
+  # We have to be careful which additional DNS RRs we cache.  For
+  # instance, PTR records are controlled by the connecting IP, and they
+  # could poison our local cache with bogus A and MX records.  
+  # Each entry is a tuple of (query_type,rr_type).  So for instance,
+  # the entry ('MX','A') says it is safe (for milter purposes) to cache
+  # any 'A' RRs found in an 'MX' query.
+  SAFE2CACHE = frozenset((
+    ('MX','MX'), ('MX','A'),
+    ('CNAME','CNAME'), ('CNAME','A'),
+    ('A','A'),
+    ('AAAA','AAAA'),
+    ('PTR','PTR'),
+    ('NS','NS'), ('NS','A'),
+    ('TXT','TXT'),
+    ('SPF','SPF')
+  ))
+
+  ## Cached DNS lookup.
+  # @param name the DNS label to query
+  # @param qtype the query type, e.g. 'A'
+  # @param cnames tracks CNAMES already followed in recursive calls
+  def dns(self, name, qtype, cnames=None):
+    """DNS query.
+
+    If the result is in cache, return that.  Otherwise pull the
+    result from DNS, and cache ALL answers, so additional info
+    is available for further queries later.
+
+    CNAMEs are followed.
+
+    If there is no data, [] is returned.
+
+    pre: qtype in ['A', 'AAAA', 'MX', 'PTR', 'TXT', 'SPF']
+    post: isinstance(__return__, types.ListType)
+    """
+    result = self.cache.get( (name, qtype) )
+    cname = None
+
+    if not result:
+        safe2cache = Session.SAFE2CACHE
+        for k, v in DNSLookup(name, qtype):
+            if k == (name, 'CNAME'):
+                cname = v
+            if (qtype,k[1]) in safe2cache:
+                self.cache.setdefault(k, []).append(v)
+        result = self.cache.get( (name, qtype), [])
+    if not result and cname:
+        if not cnames:
+            cnames = {}
+        elif len(cnames) >= MAX_CNAME:
+            #return result    # if too many == NX_DOMAIN
+            raise DNSError('Length of CNAME chain exceeds %d' % MAX_CNAME)
+        cnames[name] = cname
+        if cname in cnames:
+            raise DNSError, 'CNAME loop'
+        result = self.dns(cname, qtype, cnames=cnames)
+    return result
+
+def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
+    try:
+	# FIXME: To be thread safe, we create a fresh DnsRequest with
+	# each call.  It would be more efficient to reuse
+	# a req object stored in a Session.
+        req = DNS.DnsRequest(name, qtype=qtype, timeout=timeout)
+        resp = req.req()
+        #resp.show()
+        # key k: ('wayforward.net', 'A'), value v
+        # FIXME: pydns returns AAAA RR as 16 byte binary string, but
+        # A RR as dotted quad.  For consistency, this driver should
+        # return both as binary string.
+        #
+        if resp.header['tc'] == True:
+          if not tcpfallback:
+              raise DNS.DNSError, 'DNS: Truncated UDP Reply, SPF records should fit in a UDP packet'
+          try:
+              req = DNS.DnsRequest(name, qtype=qtype, protocol='tcp',
+                        timeout=timeout)
+              resp = req.req()
+          except DNS.DNSError, x:
+              raise DNS.DNSError, 'TCP Fallback error: ' + str(x)
+        return [((a['name'], a['typename']), a['data']) for a in resp.answers]
+    except IOError, x:
+        raise DNS.DNSError, 'DNS: ' + str(x)
+
+def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=30):
+  retVal = []
+  try:
+    # FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
+    answers = dns.resolver.query(name, qtype)
+    for rdata in answers:
+      if qtype == 'A' or qtype == 'AAAA':
+        retVal.append(((name, qtype), rdata.address))
+      elif qtype == 'MX':
+        retVal.append(((name, qtype), (rdata.preference, rdata.exchange)))
+      elif qtype == 'PTR':
+        retVal.append(((name, qtype), rdata.target.to_text(True)))
+      elif qtype == 'TXT' or qtype == 'SPF':
+        retVal.append(((name, qtype), rdata.strings))
+  except dns.resolver.NoAnswer:
+    pass
+  except dns.resolver.NXDOMAIN:
+    pass
+  return retVal
+
+try:
+    # prefer dnspython (the more complete library)
+    import dns
+    import dns.resolver  # http://www.dnspython.org
+    import dns.exception
+
+    if not hasattr(dns.rdatatype,'SPF'):
+      # patch in type99 support
+      dns.rdatatype.SPF = 99
+      dns.rdatatype._by_text['SPF'] = dns.rdatatype.SPF
+
+    DNSLookup = DNSLookup_dnspython
+except:
+    import DNS    # http://pydns.sourceforge.net
+
+    if not hasattr(DNS.Type, 'SPF'):
+        # patch in type99 support
+        DNS.Type.SPF = 99
+        DNS.Type.typemap[99] = 'SPF'
+        DNS.Lib.RRunpacker.getSPFdata = DNS.Lib.RRunpacker.getTXTdata
+
+    # Fails on Mac OS X? Add domain to /etc/resolv.conf
+    DNS.DiscoverNameServers()
+
+    DNSLookup = DNSLookup_pydns
+
+if __name__ == '__main__':
+  import sys
+  s = Session()
+  for n,t in zip(*[iter(sys.argv[1:])]*2):
+    print n,t
+    print s.dns(n,t)

dkimpy_milter/util.py

@@ -16,43 +16,81 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-def drop_privileges(milterconfig):
-    import os
+
+def fold(header):
+    """Fold a header line into multiple crlf-separated lines at column 72.
+    Borrowed from dkimpy and updated to only add \n instead of \r\n because
+    that's what the milter protocol wants.
+
+    >>> text(fold(b'foo'))
+    'foo'
+    >>> text(fold(b'foo  '+b'foo'*24).splitlines()[0])
+    'foo  '
+    >>> text(fold(b'foo'*25).splitlines()[-1])
+    ' foo'
+    >>> len(fold(b'foo'*25).splitlines()[0])
+    72
+    """
+    i = header.rfind(b"\r\n ")
+    if i == -1:
+        pre = b""
+    else:
+        i += 3
+        pre = header[:i]
+        header = header[i:]
+    maxleng = 72
+    while len(header) > maxleng:
+        i = header[:maxleng].rfind(b" ")
+        if i == -1:
+            j = maxleng
+        else:
+            j = i + 1
+        pre += header[:j] + b"\n   "
+        header = header[j:]
+    return pre + header
+
+
+def user_group(userid):
+    """Return user and group from UserID"""
     import grp
     import pwd
-    import syslog
 
-    if os.getuid() != 0:
-        if milterconfig.get('Syslog'):
-            syslog.syslog('drop_privileges: Not running as root. Cannot drop permissions.')
-        return
-
-    # Figure out if user and group are specified
-    userstr = milterconfig.get('UserID')
-    userlist = userstr.split(':')
+    userlist = userid.split(':')
     if len(userlist) == 1:
         gidname = userlist[0]
     else:
         gidname = userlist[1]
-    uidname = userlist[0]
-
     # Get the uid/gid from the name
-    running_uid = pwd.getpwnam(uidname).pw_uid
+    running_uid = pwd.getpwnam(userlist[0]).pw_uid
     running_gid = grp.getgrnam(gidname).gr_gid
+    return running_uid, running_gid
+
+
+def drop_privileges(milterconfig):
+    import os
+    import syslog
+
+    if os.getuid() != 0:
+        if milterconfig.get('Syslog'):
+            syslog.syslog('drop_privileges: Not root. No action taken.')
+        return
+
+    # Get user and group
+    uid, gid = user_group(milterconfig.get('UserID'))
 
     # Remove group privileges
     os.setgroups([])
 
     # Try setting the new uid/gid
-    os.setgid(running_gid)
-    os.setuid(running_uid)
+    os.setgid(gid)
+    os.setuid(uid)
 
     # Set umask
     old_umask = os.umask(milterconfig.get('UMask'))
 
-#################
+
 class ExceptHook:
-    def __init__(self, useSyslog = 1, useStderr = 0):
+    def __init__(self, useSyslog=1, useStderr=0):
         self.useSyslog = useSyslog
         self.useStderr = useStderr
 
@@ -68,12 +106,11 @@ class ExceptHook:
                 sys.stderr.write(line)
 
 
-####################
 def setExceptHook():
     import sys
-    sys.excepthook = ExceptHook(useSyslog = 1, useStderr = 1)
+    sys.excepthook = ExceptHook(useSyslog=1, useStderr=1)
+
 
-####################
 def write_pid(milterconfig):
     """Write PID in pidfile.  Will not overwrite an existing file."""
     import os
@@ -83,17 +120,42 @@ def write_pid(milterconfig):
         try:
             f = open(milterconfig.get('PidFile'), 'w')
         except IOError as e:
-            if milterconfig.get('Syslog'):
-                syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'.format(milterconfig.get('PidFile'), e))
-            raise
+            if str(e)[:35] == '[Errno 2] No such file or directory':
+                piddir = milterconfig.get('PidFile').rsplit('/', 1)[0]
+                os.mkdir(piddir)
+                user, group = user_group(milterconfig.get('UserID'))
+                os.chown(piddir, user, group)
+                f = open(milterconfig.get('PidFile'), 'w')
+                if milterconfig.get('Syslog'):
+                    syslog.syslog('PID dir created: {0}'.format(piddir))
+            else:
+                if milterconfig.get('Syslog'):
+                    syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'
+                                  .format(milterconfig.get('PidFile'), e))
+                raise
         f.write(pid)
         f.close()
+        user, group = user_group(milterconfig.get('UserID'))
+        os.chown(milterconfig.get('PidFile'), user, group)
     else:
         if milterconfig.get('Syslog'):
-            syslog.syslog('Unable to write pidfle {0}.  File exists.'.format(milterconfig.get('PidFile')))
-        raise RuntimeError('Unable to write pidfle {0}.  File exists.'.format(milterconfig.get('PidFile')))
+            syslog.syslog('Unable to write pidfle {0}.  File exists.'
+                          .format(milterconfig.get('PidFile')))
+        raise RuntimeError('Unable to write pidfle {0}.  File exists.'
+                           .format(milterconfig.get('PidFile')))
+    return pid
+
+
+def own_socketfile(milterconfig):
+    """If socket is Unix socket, chown to UserID before dropping privileges"""
+    import os
+    user, group = user_group(milterconfig.get('UserID'))
+    if milterconfig.get('Socket')[:1] == '/':
+        os.chown(milterconfig.get('Socket')[1:], user, group)
+    if milterconfig.get('Socket')[:6] == "local:":
+        os.chown(milterconfig.get('Socket')[6:], user, group)
+
 
-####################
 def read_keyfile(milterconfig, keytype):
     """Read private key from file."""
     import syslog
@@ -106,7 +168,8 @@ def read_keyfile(milterconfig, keytype):
         keylist = f.readlines()
     except IOError as e:
         if milterconfig.get('Syslog'):
-            syslog.syslog('Unable to read keyfile {0}.  IOError: {1}'.format(keyfile, e))
+            syslog.syslog('Unable to read keyfile {0}.  IOError: {1}'
+                          .format(keyfile, e))
         raise
     f.close()
     key = ''

etc/dkimpy-milter.conf

@@ -11,12 +11,12 @@ UMask			007
 
 # Sign for example.com with key in /etc/dkimkeys/dkim.key using
 # selector '2007' (e.g. 2007._domainkey.example.com)
-Domain			example.com
-KeyFile			/etc/mail/dkim.key
-Selector		default
+#Domain			example.com
+#KeyFile			/etc/mail/dkim.key
+#Selector		default
 
 # Commonly-used options; the commented-out versions show the defaults.
-#Canonicalization	simple
+#Canonicalization	relaxed/simple
 #Mode			sv
 
 # Socket local:/var/run/dkimpy-milter/dkimpy-milter.sock

man/dkimpy-milter.8

@@ -0,0 +1,295 @@
+\"
+.\" Standard preamble:
+.\" ========================================================================
+.de Sh \" Subsection heading
+.br
+.if t .Sp
+.ne 5
+.PP
+\fB\\$1\fR
+.PP
+..
+.de Sp \" Vertical space (when we can't use .PP)
+.if t .sp .5v
+.if n .sp
+..
+.de Vb \" Begin verbatim text
+.ft CW
+.nf
+.ne \\$1
+..
+.de Ve \" End verbatim text
+.ft R
+.fi
+..
+.\" Set up some character translations and predefined strings.  \*(-- will
+.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
+.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
+.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
+.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
+.\" nothing in troff, for use with C<>.
+.tr \(*W-
+.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
+.ie n \{\
+.    ds -- \(*W-
+.    ds PI pi
+.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
+.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
+.    ds L" ""
+.    ds R" ""
+.    ds C` ""
+.    ds C' ""
+'br\}
+.el\{\
+.    ds -- \|\(em\|
+.    ds PI \(*p
+.    ds L" ``
+.    ds R" ''
+'br\}
+.\"
+.\" If the F register is turned on, we'll generate index entries on stderr for
+.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index
+.\" entries marked with X<> in POD.  Of course, you'll have to process the
+.\" output yourself in some meaningful fashion.
+.if \nF \{\
+.    de IX
+.    tm Index:\\$1\t\\n%\t"\\$2"
+..
+.    nr % 0
+.    rr F
+.\}
+.\"
+.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
+.\" way too many mistakes in technical documents.
+.hy 0
+.if n .na
+.\"
+.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
+.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
+.    \" fudge factors for nroff and troff
+.if n \{\
+.    ds #H 0
+.    ds #V .8m
+.    ds #F .3m
+.    ds #[ \f1
+.    ds #] \fP
+.\}
+.if t \{\
+.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
+.    ds #V .6m
+.    ds #F 0
+.    ds #[ \&
+.    ds #] \&
+.\}
+.    \" simple accents for nroff and troff
+.if n \{\
+.    ds ' \&
+.    ds ` \&
+.    ds ^ \&
+.    ds , \&
+.    ds ~ ~
+.    ds /
+.\}
+.if t \{\
+.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
+.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
+.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
+.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
+.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
+.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
+.\}
+.    \" troff and (daisy-wheel) nroff accents
+.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
+.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
+.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
+.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
+.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
+.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
+.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
+.ds ae a\h'-(\w'a'u*4/10)'e
+.ds Ae A\h'-(\w'A'u*4/10)'E
+.    \" corrections for vroff
+.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
+.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
+.    \" for low resolution devices (crt and lpr)
+.if \n(.H>23 .if \n(.V>19 \
+\{\
+.    ds : e
+.    ds 8 ss
+.    ds o a
+.    ds d- d\h'-1'\(ga
+.    ds D- D\h'-1'\(hy
+.    ds th \o'bp'
+.    ds Th \o'LP'
+.    ds ae ae
+.    ds Ae AE
+.\}
+.rm #[ #] #H #V #F C
+.\" ========================================================================
+.\"
+.IX Title "dkimpy-milter 8"
+.TH dkimpyy-milter 8
+.SH NAME
+.B dkimpy
+\- DKIM signing and verifying filter for MTAs
+.SH SYNOPSIS
+.B dkimpy-milter [configfile]
+
+.SH DESCRIPTION
+.B dkimpy-milter
+implements the 
+.B DKIM
+standard for signing and verifying e-mail messages on a per-domain basis.
+
+.B dkimpy-milter
+uses the
+.I milter
+interface, originally distributed as part of version 8.11 of
+.B sendmail(8),
+to provide DKIM signing and/or verifying service for mail transiting
+a milter-aware MTA.
+
+.SH DATA SETS
+Many of the configuration file parameters will refer to a "dataset" as their
+values.  This refers to a string that either contains the list of desirable
+values, or to a file that contains them, or a database containing the data.
+
+Some data sets require that the value contain more than one entry.  How this
+is done depends on which data set type is used.  Not all these datasets are
+currently used by dkimpy-milter.  See
+.B dkimpy-milter.conf(5)
+for details about specific options and which dataset types they use.
+
+In particular:
+.TP
+.I a)
+If the string begins with "file:", then the remainder of the string is
+presumed to refer to a flat file that contains elements of the data set,
+one per line.  If a line contains whitespace-separated values, then the
+line is presumed to define a key and its corresponding value.  Blank lines
+are ignored, and the hash ("#") character denotes the start of a comment.
+If a value contains multiple entries, the entries should be separated by
+colons.
+.TP
+.I c)
+If the string begins with "db:" and the program was compiled with
+Sleepycat DB support, then the remainder of the string is presumed to
+identify a Sleepycat database containing keys and corresponding values.
+These may be used only to test for membership in the data set, or for
+storing keys and corresponding values.  If a value contains multiple entries,
+the entries should be separated by colons. [Not implemented yet]
+.TP
+.I h)
+If the string contains none of these prefixes but ends with ".db", it
+is presumed to be a Sleepycat DB as described above (if support for same
+is compiled in).  [Not implemented yet]
+.TP
+.I i)
+If the string contains none of these prefixes but starts with a slash ("/")
+character, it is presumed to be a flat file as described above.
+.TP
+.I j)
+If the string begins with "csl:", the string is treated as a comma-separated
+list as described in m) below.
+.TP
+.I l)
+If the string begins with "mdb:", it refers to a directory that contains
+a memory database, as provided by libmdb from OpenLDAP.  [Not implemented yet]
+.TP
+.I m)
+In any other case, the string is presumed to be a comma-separated list.
+Elements in the list are either simple data elements that are part of the
+set or, in the case of an entry of the form "x=y", are stored as key-value
+pairs as described above.
+.SH OPTIONS
+.TP
+See
+.I dkimpy-milter.conf(5)
+information about available options.  Unlike OpenDKIM, dkimpy-milter does not
+support command line option switches.
+
+When signing a message, a
+.I DKIM-Signature:
+header will be prepended to the message.  The signature is computed using
+the private key provided.  You must be running a version of
+.I sendmail(8)
+recent enough to be able to do header prepend operations (8.13.0 or later).
+
+When verifying a message, an
+.I Authentication-Results:
+header will be prepended to indicate the presence of a signature and whether
+or not it could be validated against the body of the message using the
+public key advertised by the sender's nameserver.  The value of this header
+can be used by mail user agents to sort or discard messages that were not
+signed or could not be verified.
+
+.SH FILE PERMISSIONS
+When the filter is started as the superuser and the UserID setting is
+used, the filter gives up its root privileges by changing to the specified
+user after the following steps are taken: (1) the configuration file (if any)
+is loaded; (2) if the KeyFile or KeyFileEd25519 settings are used, the keys are
+loaded into memory; (3) all data sets in the configuration file are opened, and
+those that are based on flat files are also read into memory; and (4) if
+ChangeRootDirectory is set, the process root is changed to that directory.
+This means on configuration reload, the filter will not be accessing these
+files or the configuration file as the superuser (and possibly from a
+different root), and any key files referenced by the KeyTable will also be
+accessed by the new user.
+
+Thus, keys referenced by the KeyTable must always be accessible for read by
+the unprivileged user.  Also, run-time reloads are not possible if any of the
+other files will not be readable by the unprivileged user.
+.SH ENVIRONMENT
+The following environment variable(s) can be used to adjust the behaviour
+of this filter:
+.TP
+.I DKIM_TMPDIR
+The directory to use when creating temporary files.  The default is
+.I /tmp.
+.SH NOTES
+When using DNS timeouts be sure not to use a timeout that is larger than the
+timeout being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.
+
+Features that involve specification of IPv4 addresses or CIDR blocks
+will use the
+.I inet_addr(3)
+function to parse that information.  Users should be familiar with the
+way that function handles the non-trivial cases (for example, "192.0.2/24"
+and "192.0.2.0/24" are not the same thing).
+.SH EXIT STATUS
+Filter exit status codes are selected according to
+.I sysexits(3).
+.SH HISTORY
+DKIM is an amalgam of Yahoo!'s
+.B DomainKeys
+proposal, and Cisco's
+.B Internet Identified Mail
+(IIM) proposal.
+.SH VERSION
+This man page covers version 0.9.4 of
+.I dkimpy-milter.
+.SH COPYRIGHT
+Copyright (c) 2005-2008, Sendmail, Inc. and its suppliers.  All rights
+reserved.
+
+Copyright (c) 2009-2013, 2015, The Trusted Domain Project.
+All rights reserved.
+
+Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>
+.SH SEE ALSO
+.I dkimpy-milter.conf(5), sendmail(8)
+.P
+Sendmail Operations Guide
+.P
+RFC5321 - Simple Mail Transfer Protocol
+.P
+RFC5322 - Internet Messages
+.P
+RFC6376 - DomainKeys Identified Mail
+.P
+RFC7601 - Message Header Field for Indicating Message Authentication Status
+.P
+draft-ietf-dcrup-dkim-crypto - A new cryptographic signature method for DKIM

man/dkimpy-milter.conf.5

@@ -133,7 +133,7 @@
 dkimpy-milter \- Python milter for DKIM signing and validation
 .SH "VERSION"
 .IX Header "VERSION"
-0\.9\.1
+0\.9\.2
 
 .SH "DESCRIPTION"
 .IX Header "DESCRIPTION"
@@ -208,23 +208,19 @@ the canonicalization method.  The recognized values are
 and
 .I simple
 as defined by the DKIM specification.  The default is
+.I relaxed
+/
 .I simple.
 The value may include two different canonicalizations separated by a
 slash ("/") character, in which case the first will be applied to the
 header and the second to the body.
 
-.TP
-.I Diagnostics (Boolean)
-Requests the inclusion of "z=" tags in signatures, which encode the
-original header field set for use by verifiers when diagnosing verification
-failures.  Not recommended for normal operation. [dkimpy-milter specific: also
-increases the verbosity of Syslog logging if enabled.]
-
 .TP
 .I DiagnosticDirectory (string)
 Directory into which to write diagnostic reports when message verification
-fails on a message bearing a "z=" tag.  If not set (the default), these files
-are not generated.
+fails.  If not set (the default), these files are not generated.  [Unlike
+OpenDKIM, this applies to all messages, not just  on messages bearing a "z=" tag
+because dkimpy does not yet support "z=".]
 
 .TP
 .I Domain (dataset)
@@ -234,11 +230,11 @@ domains will be verified rather than being signed.
 This parameter is not required if a
 .I SigningTable
 is in use; in that case, the list of signed domains is implied by the
-lines in that file. [NOT IMPLEMENTED]
+lines in that file. [SigningTable NOT IMPLEMENTED]
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [NOT IMPLEMENTED]
+is defined. [KeyTable NOT IMPLEMENTED]
 
 .TP
 .I InternalHosts (dataset)
@@ -259,13 +255,50 @@ all messages.  Ignored if a
 is defined. [KeyTable NOT IMPLEMENTED]
 
 .TP
-.I KeyFileEd25119 (string)
+.I KeyFileEd25519 (string)
 Gives the location of a Ed25519 private key to be used for Ed25519 signing
 all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
 generation (as used in dkimpy).  Ignored if a 
 .I KeyTableEd25519
 is defined.  [KeyTableEd25519 NOT IMPLEMENTED]
 
+.TP
+.I MacroList (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be a
+local user and therefore whether or not the message should be signed.  If
+a
+.I value
+is specified matching a macro name in the data set, the value of the macro
+must match a value specified (matching is case-sensitive), otherwise the
+macro must be defined but may contain any value.  The set is empty by
+default, meaning macros are not considered when making the sign-verify
+decision.  The general format of the value is
+.I value1[|value2[|...]];
+if one or more value is defined then the macro must be set to one of the
+listed values, otherwise the macro must be set but can contain any
+value.
+
+In order for the macro and its value to be available to the filter for
+checking, the MTA must send it during the protocol exchange.  This is either
+accomplished via manual configuration of the MTA to send the desired macros
+or, for MTA/filter combinations that support the feature, the filter can
+request those macros that are of interest.  The latter is a feature negotiated
+at the time the filter receives a connection from the MTA and its availability
+depends upon the version of milter used to compile the filter and the version
+of the MTA making the connection.
+
+.TP
+.I MacroListVerify (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be an
+external source and therefore whether or not the message should be signed.
+Entries in this data set follow the same form as those of the
+.I MacroList
+option above.  [this option is not inhereted from OpenDKIM]  
+
 .TP
 .I Mode (string)
 Selects operating modes.  The string is a concatenation of characters that
@@ -372,7 +405,7 @@ Log via calls to
 using the named facility.  The facility names are the same as the ones
 allowed in
 .I syslog.conf(5).
-The default is "mail".  [Hardcoded to default for now]
+The default is "mail".
 
 .TP
 .I SyslogSuccess (Boolean)

setup.py

@@ -18,19 +18,27 @@
 
 from setuptools import setup
 import os
+import dkimpy_milter
 
 description = "Domain Keys Identified Mail (DKIM) signing/verifying milter for Postfix/Sendmail."
 
+kw = {}  # Work-around for lack of 'or' requires in setuptools.
+try:
+    import DNS
+    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'PyDNS']
+except ImportError:  # If PyDNS is not installed, prefer dnspython
+    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'dnspython']
+
 setup(
     name='dkimpy-milter',
-    version='0.9.1',
+    version=dkimpy_milter.__version__,
     author='Scott Kitterman',
     author_email='scott@kitterman.com',
     url='https://launchpad.net/dkimpy-milter',
     description=description,
     download_url = "https://pypi.python.org/pypi/dkimpy-milter",
     classifiers= [
-        'Development Status :: 3 - Alpha',
+        'Development Status :: 4 - Beta',
         'Environment :: No Input/Output (Daemon)',
         'Intended Audience :: System Administrators',
         'License :: OSI Approved :: GNU General Public License (GPL)',
@@ -49,8 +57,11 @@ setup(
     },
     include_package_data=True,
     data_files=[(os.path.join('share', 'man', 'man5'),
-        ['man/dkimpy-milter.conf.5']), ('etc', ['etc/dkimpy-milter.conf'])],
-
-    install_requires = ['dkimpy', 'pymilter', 'authres>=1.0.2'],
+        ['man/dkimpy-milter.conf.5']), (os.path.join('share', 'man', 'man8'),
+        ['man/dkimpy-milter.8']), ('etc', ['etc/dkimpy-milter.conf']),
+        (os.path.join('lib', 'systemd', 'system'),
+        ['system/dkimpy-milter.service']),(os.path.join('etc', 'init.d'),
+        ['system/dkimpy-milter'])],
     zip_safe = False,
+    **kw
 )

system/dkimpy-milter

@@ -0,0 +1,133 @@
+#! /bin/sh
+#
+# skeleton	example file to build /etc/init.d/ scripts.
+#		This file should be used to construct scripts for /etc/init.d.
+#
+#		Written by Miquel van Smoorenburg <miquels@cistron.nl>.
+#		Modified for Debian 
+#		by Ian Murdock <imurdock@gnu.ai.mit.edu>.
+#
+# Version:	@(#)skeleton  1.9  26-Feb-2001  miquels@cistron.nl
+#
+### BEGIN INIT INFO
+# Provides:          dkim-milter dkim-milter-python dkimpy-milter
+# Required-Start:    $remote_fs $syslog $network $time
+# Required-Stop:     $remote_fs $syslog $network
+# Should-Start:
+# Should-Stop:
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: dkimpy-milter
+# Description:       Python DKIM Milter for Sendmail and Postfix
+### END INIT INFO
+prefix="/usr/local"
+exec_prefix=${prefix}
+sysconfdir="/etc/dkimpy-milter"
+bindir="${exec_prefix}/bin/"
+RUNDIR="/var/run/dkimpy-milter"
+DAEMON=${bindir}/dkimpy-milter
+PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
+NAME=dkimpy-milter
+DESC="Python DKIM Milter"
+USER=dkimpy-milter
+GROUP=dkimpy-milter
+SOCKET=$RUNDIR/dkimpy-milter.pid
+
+test -x $DAEMON || exit 0
+
+# Include dkimpy-python defaults if available
+# Typically not used
+if [ -f /etc/default/dkimpy-milter ] ; then
+	. /etc/default/dkimpy-milter
+fi
+
+set -e
+
+. /lib/lsb/init-functions
+
+case "$1" in
+  start)
+	echo -n "Starting $DESC: "
+	# Create the run directory if it doesn't exist
+	if [ ! -d $RUNDIR ]; then
+		install -o $USER -g $GROUP -m 755 -d $RUNDIR || return 2
+	fi
+
+	# Clean up stale sockets
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		pid=`cat $RUNDIR/$NAME.pid`
+		if ! ps -C $DAEMON -s $pid >/dev/null; then
+			rm $RUNDIR/$NAME.pid
+			# UNIX sockets may be specified with or without the
+			# local: prefix; handle both
+			t=`echo $SOCKET | cut -d: -f1`
+			s=`echo $SOCKET | cut -d: -f2`
+			if [ -e $s -a -S $s ]; then
+				if [ "$t" = "$s" -o "$t" = "local" ]; then
+					rm $s
+				fi
+			fi
+		fi
+	fi
+
+	start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+		$RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+	echo "$NAME."
+	;;
+  stop)
+	echo -n "Stopping $DESC: "
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid 
+		rm $RUNDIR/$NAME.pid
+		#echo $SOCKET
+		if [ -e $SOCKET ]; then
+			rm $SOCKET
+		fi
+	fi
+	echo "$NAME."
+	;;
+  force-reload)
+        echo -n "Force reloading $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        sleep 1
+        start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+        ;;
+  restart)
+        echo "Restarting $DESC: "
+        echo -n "Stopping $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        echo "$NAME."
+	sleep 1
+        echo -n "Starting $DESC: "
+        start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+	;;
+  status)
+        status_of_proc -p /var/run/dkimpy-milter/dkimpy-milter.pid /usr/local/bin/dkimpy-milter dkimpy-milter
+        ;;
+
+  *)
+	N=/etc/init.d/$NAME
+	echo "Usage: $N {start|stop|force-reload|restart|}" >&2
+	exit 1
+	;;
+esac
+
+exit 0

system/dkimpy-milter.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+After=syslog.target network.target
+
+[Service]
+Type=simple
+PIDFile=/var/run/dkimpy-milter/dkimpy-milter.pid
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter.conf 
+
+[Install]
+WantedBy=multi-user.target