Set release date

service files are not shipped in the sdist

.gitignore

@@ -1,3 +1,4 @@
 dist
 dkimpy_milter.egg-info
 *.pyc
+*~

CHANGES

@@ -1,22 +1,53 @@
-1.0.3 2019-11-22
- - Make error logging more explicit to aid debugging
- - Delete own_socketfile to resolve race condition where the permissions
-   change fails on a Unix socket because it hasn't been created yet (libmilter
-   will do this correctly on its own based on umask, the milter doesn't need
-   to do it) (LP: #1849712)
+1.2.1 2020-01-04
+ - Fix expand option not to fail if files are missing since socket activation
+   service files are not shipped in the sdist
+ - Correct dkimpy-milter.conf file install location to match expand locations
 
-1.0.2 2019-10-07
+1.2.0 2020-01-03
+ - Add support for SigningTable, KeyTable, and KeyTableEd25519 (LP: #1797397)
+ - Add support for specifying MinimumKeyBits for RSA signatures
+ - Add support for SignHeaders feature, thanks to Ralph Seichter for the patch
+ - Add support for specifying DNSTimeout (bumps required dkimpy version to 1.0)
+ - Add information on message content conversion to README
+ - Add new expand option to setup.py so various file system locations can be
+   specified at build/install time rather than being hard coded
+ - Install openrc init file for Gentoo and other openrc users
+ - Add support for passing PID file name on command line to make it easier to
+   keep system init and daemon configuration in sync
+ - Add support for storing DKIM failed mails in a specified
+   DiagnosticDirectory
  - Fix startup logging so it provides information at a useful time
+ - Fix verify processing so missing (optional) i= tag doesn't cause the milter
+   to fail  (LP: #1842250)
  - Fix message extraction so that signing in the same pass through the milter
    as verifying works correctly
+ - Add debug logging for content type to assist troubleshooting MIME
+   conversion issues
  - Fix variable initialization so mailformed mails missing body From do not
    cause a traceback (LP: #1844161)
  - Catch more ascii encoding errors to improve resilience against bad data
    (LP: #1844189)
  - Fix sysv init so it works (LP: #1839487)
+ - Make error logging more explicit to aid debugging
+ - Remove SigningTableEd25519 from documentation - it was never implemented
+   and a per algorithm signing table turns out not to be needed
+ - Delete own_socketfile to resolve race condition where the permissions
+   change fails on a Unix socket because it hasn't been created yet (libmilter
+   will do this correctly on its own based on umask, the milter doesn't need
+   to do it) (LP: #1849712)
+
+1.1.0 2019-04-12
+ - Add SubDomains option to enable signing for sub-domains (LP: #1811535)
+ - Port to python3 (LP: #1815502)
+ - Add test suite using opendkim miltertest
+ - When Socket is absolute path, do not strip leading /
+ - Handle unix: socket prefix the same as local:
+ - Set up correct AuthservID defaults
+ - config: Reassemble strings sensibly
+ - Consistently prefer dnspython to Py3DNS (LP: #1815558)
 
 1.0.1 2019-02-11
- * Reorder milter start and dropping privileges so permissions on Unix socket
+ - Reorder milter start and dropping privileges so permissions on Unix socket
    are correct (LP: 1797720)
  - Make domain checks case insensitive for determining if signing should be
    done (LP: #1815311)

README.md

@@ -1,5 +1,4 @@
-OVERVIEW
-========
+# OVERVIEW
 
 This is a DKIM signing and verification milter.  It has been tested with both
 Postfix and Sendmail.
@@ -9,21 +8,28 @@ a subset of OpenDKIM options are supported.  If an unsupported option is
 specified, an error will be raised.
 
 
-INSTALLATION
-===========
+# INSTALLATION
 
 This package includes a default configuration file and man pages.  For those
 to be installed when installing using setup.py, the following incantation is
 required because setuptools developers decided not being able to do this by
 default is a feature:
 
-python setup.py install --single-version-externally-managed --record=/dev/null
+[sudo] python3 setup.py install --single-version-externally-managed --record=/dev/null
 
 For users of Debian Stable (Debian 9, Codename Squeeze), all dependencies are
 available in either the main or backports repositories:
 
-[sudo] apt install python-milter python-nacl python-ipaddress python-dnspython
-[sudo] apt install -t stretch-backports python-authres python-dkim
+[sudo] apt install python3-milter python3-nacl python3-dnspython
+[sudo] apt install -t stretch-backports python3-authres python3-dkim
+
+It is also available in the Debian package archive:
+
+[sudo] apt install dkimpy-milter [Debian 10 or later]
+[sudo] apt install -t stretch-backports dkimpy-milter [Debian 9]
+
+When installing using the Debian package, all dependencies are automatically
+installed.
 
 The preferred method of installation is from PyPi using pip (if distribution
 packages are not available):
@@ -33,27 +39,48 @@ packages are not available):
 Using pip will cause required packages to be installed via easy_install if they
 have not been previously installed.  Because pymilter and PyNaCl are compiled
 Python extensions, the system will need appropriate development packages and
-an C compiler.  Alternately, install these dependencies from dsitribution/OS
+an C compiler.  Alternately, install these dependencies from distribution/OS
 packages and then pip install dkimpy_milter.
 
-The milter will work with either pydns (DNS) or dnspython (dns), preferring
+The milter will work with either py3dns (DNS) or dnspython (dns), preferring
 dnspython if both are available.  The dkimpy DKIM module also works with
 either.
 
+## NON-STANDARD INSTALLATION PATHS
 
-SETUP
-====
+The package includes a custom setup command called expand.  It allows various
+file locations in init scripts, man pages, and config files to be over-ridden
+at install time.
 
-SIGNING KEYS
-============
+expand: Expand @@ variables in input files, simlar to make macros.
+user_options:
+ --sysconfigdir=, e: Specify system configuration directory.
+ --sbindir=, s: Specify system binary directory [not used].
+ --bindir=, b: Specify binary directory.
+ --rundir=,r: Specify run state directory.
+
+As an example, to change the run directory to /var/run, one would do:
+
+python3 setup.py expand --rundir=/var/run
+[sudo] python3 setup.py install --single-version-externally-managed \
+                                --record=/dev/null
+
+or in a single step (the order matters):
+
+[sudo] python3 setup.py expand --rundir=/var/run install \
+                               --single-version-externally-managed \
+                               --record=/dev/null
+
+# SETUP
+
+## SIGNING KEYS
 
 In order to create DKIM signatures, a private key must be available.  Signing
 keys should be protected (owned by root:root with permissions 600 in a
 directory that is not world readable).  Different keys are required for RSA
 and (if used) Ed25519.
 
-RSA
-===
+### RSA
 
 Both public and private keys for RSA have standard formats and there are many
 tools available to create them.  Keys must (RFC 8302) have a minimum size of
@@ -66,8 +93,7 @@ will produce both the private key file (.key suffix) and a file with the DKIM
 public key record to be published DNS (.dns suffix).  RSA is the default key
 type.  2048 bits is the default key size.
 
-ED25519
-=======
+### ED25519
 
 There is no standardized non-binary representation for Ed25519 private keys,
 so in order to generate Ed25519 keys for dkimpy-milter, dkimpy specific tools
@@ -79,8 +105,76 @@ will provide both the private key file (.key suffix) and a file with the DKIM
 public key record to be published DNS (.dns suffix).  Ed25519 keys do not have
 variable bit lengths.
 
-MTA INTEGRATION
-==============
+### COMPLEX SIGNING CONFIGURATIONS
+
+The KeyTable, KeyTableEd25519, and SigningTable are used to define signing
+instructions to the filter where use of Domain, Selector and KeyFile together
+are insufficient.
+
+First, select the type of database you will use for each.  They need not
+be the same.  The "DATA SETS" portion of the dkimpy-milter(8) man page
+describes the possibilities and how they are formatted.  Then, construct those
+databases.
+
+Let's suppose you want to sign for two domains, example.com and example.net.
+Within example.com, you want to sign for user "president" differently than
+everyone else.  Let's say further that you want to use a flat text file.
+
+You've generated private key files for each of these and stored them
+in the directory /usr/local/etc/dkim/keys as files "president", "excom" and
+"exnet", with the obvious intents.  You want to use selectors "foo", "bar"
+and "baz" for those, respectively.  The signing domains match the senders
+(i.e. the signatures for example.com's stuff will be held by example.com,
+and example.net likewise).
+
+First, write the KeyTable.  This is a list of the keys you intend to use,
+and you just assign arbitrary names to them.  So as a flat file, the KeyTable
+for the above might look like this:
+
+	preskey	example.com:foo:/usr/local/etc/dkim/keys/president
+	comkey	example.com:bar:/usr/local/etc/dkim/keys/excom
+	netkey	example.net:baz:/usr/local/etc/dkim/keys/exnet
+
+If also signing with ed25519, specify a KeyTableEd25519 pointing to the keys
+needed for ed25519.  Both KeyTable and KeyTableEd25519 are evaluated if there
+is a SigningTable (see below).
+
+Per the documentation, multi-field data sets that are made of flat files have
+the fields separated by colons, but the key and value(s) are separated by
+whitespace.
+
+So now we've named each key file, and specified with which selector and domain
+each will be used, and then given each of those groupings a name.  This
+is your KeyTable.  Let's say you put it in /usr/local/etc/dkim/keytable.
+
+Next, write the SigningTable.  This maps senders (by default, taken from the
+From: header field of a message passing through the filter) to which keys
+will be used to sign their mail.  Wildcards are allowed.  So to do what was
+described above, we write it as follows:
+
+	president@example.com	preskey
+	*@example.com		comkey
+	*@example.net		netkey
+
+Since we want to use wildcards, we can't actually use a regular flat file.
+Wildcards require a regular expression file, or "refile".  The above is
+valid format for one of those.  Let's say you put this in
+/usr/local/etc/dkim/signingtable.
+
+Finally, tell the filter that it should use these files by adding this to
+your configuration file:
+
+	KeyTable	/usr/local/etc/dkim/keytable
+	SigningTable	refile:/usr/local/etc/dkim/signingtable
+
+You could put "file:" in front of the filename for the KeyTable just to be
+precise, but "file:" is assumed if the value starts with a "/".
+
+Note: Unlike opendkim, dkimpy-milter will check for "\*" in the signing table
+regardless of if refile is specified or not.  Use of refile is supported for
+compatibility with configurations initially developed for use with opendkim.
+
+## MTA INTEGRATION
 
 Both a systemd unit file and a sysv init file are provided.  Both make
 assumptions about defaults being used, e.g. if a non-standard pidfile name is
@@ -93,7 +187,7 @@ As an example, using the default dkimpy-user on Debian, the command would be:
 
 [sudo] adduser --system --no-create-home --quiet --disabled-password \
                --disabled-login --shell /bin/false --group \
-               --home /var/run/dkimpy-milter dkimpy-milter
+               --home /run/dkimpy-milter dkimpy-milter
 
 Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
 missing, the milter will create it on startup.
@@ -107,16 +201,20 @@ the following steps:
 [sudo] systemctl status dkimpy-milter (to verify it started correctly)
 
 As with all milters, dkimpy-milter needs to be integrated with your MTA of
-choice (Sendmail or Postfix).
+choice (Sendmail or Postfix).  When integrating with your MTA, the risk of
+signature invalidation due to content conversion of the message body needs to
+be considered.  See RFC 6376, Section 5.3 for discussion of this issue.  As a
+practical matter, when signing, configure the milter to follow all others that
+might modify the message body.  When verifying, configure the milter before
+other processes that might modify the message body.
 
-SENDMAIL
-========
+### SENDMAIL
 
 Configuration is very similar to opendkim, but needs some adjustment for
 dkimpy-milter. Here's an example configuration line to include in your
 sendmail.mc:
 
-INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/var/run/dkimpy-milter/dkimpy-milter.sock')dnl
+INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/run/dkimpy-milter/dkimpy-milter.sock')dnl
 
 Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
 and a restart of sendmail.  Note that S= needs to match the value of Socket in
@@ -126,8 +224,7 @@ Milter support should be present by default in most versions of sendmail
 these days, but if not included in your Sendmail build, see:
 http://www.elandsys.com/resources/sendmail/milter.html
 
-ISSUES USING SENDMAIL TO SIGN AND VERIFY
-========================================
+#### ISSUES USING SENDMAIL TO SIGN AND VERIFY
 
 When using the sendmail MTA in both signing and verifying mode, there are
 a few issues of which to be aware that might cause operational problems
@@ -177,8 +274,7 @@ and deserve consideration.
     in the rewritten form, guaranteeing the input and output are the same
     and thus the signature matches the payload.
 
-POSTFIX
-=======
+### POSTFIX
 
 Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
 README_FILES/MILTER_README).  Here's an example master.cf excerpt that talks
@@ -228,8 +324,7 @@ MacroListVerify		daemon_name|VERIFYING
 ...
 
 
-NOTES
-=====
+# NOTES
 
 The python DKIM library, dkimpy, requires the entire message being signed or
 verified to be in memory, so dkimpy-milter does not write messages out to a

TODO

@@ -44,9 +44,19 @@ No additional features
 1.0.1
 Bug fix only, improved documentation
 
-1.1.0 (planned)
-Port to Python 3
-Subdomain support
+1.1.0
+Port to Python 3  implemented	verified
+Subdomain support implemented	verified
+Test suite	  implemented	verified
+
+1.2.0
+DNSTimeout (dkimpy 1.0)  implemented  verified by inspection
+KeyTable            implemented  verified
+KeytableEd25519     implemented  verified
+MinimumKeyBits      implemented  verified
+SignHeaders	    implemented  verified by inspection
+SigningTable        implemented  verified
+TemporaryDirectory  implemented  verified by inspection
 
 Planned dataset type support (if needed):
 mdb:
@@ -56,9 +66,8 @@ Considered for near-term feature release
 AlwaysAddARHeader
 ChangeRootDirectory
 ClockDrift (requires dkimpy change)
-DNSTimeout (requires dkimpy change)
 MilterDebug
-MinimumKeyBits
+OmitHeaders
 OversignHeaders (may require dkimpy changes)
 PeerList
 SignatureAlgorithm
@@ -73,8 +82,6 @@ ExternalIgnoreList
 FixCRLF
 KeepAuthResults
 KeepTemporaryFiles
-KeyTable
-KeytableEd25519
 LogResults
 LogWhy
 MaximumHeaders
@@ -82,7 +89,6 @@ MaximumSignaturesToVerify
 MultipleSignatures
 MustBeSigned
 NoHeaderB 
-OmitHeaders
 On-BadSignature
 On-Default
 On-DNSError
@@ -97,12 +103,8 @@ RequiredHeaders
 RequireSafeKeys
 SignatureAlgorithm
 SignatureTTL
-SignHeaders
-SigningTable
 SoftwareHeader
 StrictHeaders
-SubDomains
-TemporaryDirectory
 TestDNSData
 TestPublicKeys
 

dkimpy_milter/__init__.py

@@ -1,4 +1,4 @@
-#! /usr/bin/python2
+#! /usr/bin/python3
 # Original dkim-milter.py code:
 # Author: Stuart D. Gathman <stuart@bmsi.com>
 # Copyright 2007 Business Management Systems, Inc.
@@ -28,17 +28,18 @@ import dkim
 import authres
 import os
 import tempfile
-import StringIO
+import io
 import re
+import codecs
 from Milter.utils import parse_addr, parseaddr
 import dkimpy_milter.config as config
 from dkimpy_milter.util import drop_privileges
 from dkimpy_milter.util import setExceptHook
 from dkimpy_milter.util import write_pid
-from dkimpy_milter.util import read_keyfile
+from dkimpy_milter.util import get_keys
 from dkimpy_milter.util import fold
 
-__version__ = "1.0.1"
+__version__ = "1.2.0"
 FWS = re.compile(r'\r?\n[ \t]+')
 
 
@@ -50,10 +51,9 @@ class dkimMilter(Milter.Base):
         self.id = Milter.uniqueID()
         # we don't want config used to change during a connection
         self.conf = milterconfig
-        self.privatersa = privateRSA
-        self.privateed25519 = privateEd25519
         self.fp = None
         self.fdomain = ''
+        self.iequals = None
 
     @Milter.noreply
     def connect(self, hostname, unused, hostaddr):
@@ -61,21 +61,23 @@ class dkimMilter(Milter.Base):
         self.external_connection = False
         self.hello_name = None
         # sometimes people put extra space in sendmail config, so we strip
-        self.receiver = self.getsymval('j').strip()
+        self.receiver = self.getsymval('j')
+        if self.receiver is not None:
+            self.receiver = self.receiver.strip()
         try:
-            self.AuthservID = milterconfig['AuthservID']
+            self.AuthservID = self.conf['AuthservID']
         except:
             self.AuthservID = self.receiver
         if hostaddr and len(hostaddr) > 0:
             ipaddr = hostaddr[0]
-            if milterconfig['IntHosts']:
-                if milterconfig['IntHosts'].match(ipaddr):
+            if self.conf['IntHosts']:
+                if self.conf['IntHosts'].match(ipaddr):
                     self.internal_connection = True
         else:
             ipaddr = ''
         self.connectip = ipaddr
-        if milterconfig.get('MacroList') and not self.internal_connection:
-            macrolist = milterconfig.get('MacroList')
+        if self.conf.get('MacroList') and not self.internal_connection:
+            macrolist = self.conf.get('MacroList')
             for macro in macrolist:
                 macroname = macro.split('|')[0]
                 macroname = '{' + macroname + '}'
@@ -83,8 +85,8 @@ class dkimMilter(Milter.Base):
                 if ((len(macro.split('|')) == 1 and macroresult) or macroresult
                         in macro.split('|')[1:]):
                     self.internal_connection = True
-        if milterconfig.get('MacroListVerify'):
-            macrolist = milterconfig.get('MacroListVerify')
+        if self.conf.get('MacroListVerify'):
+            macrolist = self.conf.get('MacroListVerify')
             for macro in macrolist:
                 macroname = macro.split('|')[0]
                 macroname = '{' + macroname + '}'
@@ -96,7 +98,7 @@ class dkimMilter(Milter.Base):
             connecttype = 'INTERNAL'
         else:
             connecttype = 'EXTERNAL'
-        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 1:
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 1:
             syslog.syslog("connect from {0} at {1} {2}"
                           .format(hostname, hostaddr, connecttype))
         return Milter.CONTINUE
@@ -106,9 +108,9 @@ class dkimMilter(Milter.Base):
     # of each message.
     @Milter.noreply
     def envfrom(self, f, *str):
-        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 2:
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 2:
             syslog.syslog("mail from: {0} {1}".format(f, str))
-        self.fp = StringIO.StringIO()
+        self.fp = io.BytesIO()
         self.mailfrom = f
         t = parse_addr(f)
         if len(t) == 2:
@@ -123,9 +125,14 @@ class dkimMilter(Milter.Base):
     @Milter.noreply
     def header(self, name, val):
         lname = name.lower()
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4:
+            if lname == 'content-transfer-encoding':
+                syslog.syslog("content-transfer-encodeing: {0}".format(val))
+            if lname == 'content-type':
+                syslog.syslog("content-type: {0}".format(val))
         if lname == 'dkim-signature':
-            if (milterconfig.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 1):
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog("{0}: {1}".format(name, val))
             self.has_dkim += 1
         if lname == 'from':
@@ -134,23 +141,23 @@ class dkimMilter(Milter.Base):
                 self.fdomain = self.author.split('@')[1].lower()
             except IndexError as er:
                 pass # self.author was not a proper email address
-            if (milterconfig.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 1):
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog("{0}: {1}".format(name, val))
         elif lname == 'authentication-results':
             self.arheaders.append(val)
         if self.fp:
             try:
-                self.fp.write("%s: %s\n" % (name, val))
+                self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'ascii')))
             except:
-                # Don't choke on header fields with garbage in them.
+                # Don't choke on header fields with non-ascii garbage in them.
                 pass
         return Milter.CONTINUE
 
     @Milter.noreply
     def eoh(self):
         if self.fp:
-            self.fp.write("\n")   # terminate headers
+            self.fp.write(b"\n")   # terminate headers
         self.bodysize = 0
         return Milter.CONTINUE
 
@@ -172,87 +179,179 @@ class dkimMilter(Milter.Base):
                       .parse_value(FWS.sub('', val)))
                 if ar.authserv_id == self.AuthservID:
                     self.chgheader('authentication-results', i, '')
-                    if (milterconfig.get('Syslog') and
-                            milterconfig.get('debugLevel') >= 1):
+                    if (self.conf.get('Syslog') and
+                            self.conf.get('debugLevel') >= 1):
                         syslog.syslog('REMOVE: {0}'.format(val))
             except:
                 # Don't error out on unparseable AR header fiels
                 pass
         # Check and/or sign DKIM
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4):
+            syslog.syslog('self.conf: {0}'.format(self.conf))
         self.fp.seek(0)
         txt = self.fp.read()
-        if milterconfig.get('Domain'):
-            domain = milterconfig.get('Domain')
-        else:
-            domain = ''
-        if ((self.fdomain in domain) and not milterconfig.get('Mode') == 'v'
+        self.get_identities_sign()
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3):
+            syslog.syslog('self.domain: {0}, self.fdomain: {1}, self.iequals: {2}'.format(self.domain, self.fdomain, self.iequals))
+        if ((self.fdomain in self.domain) and not self.conf.get('Mode') == 'v'
                 and not self.external_connection):
             self.sign_dkim(txt)
         if ((self.has_dkim) and (not self.internal_connection) and
-            (milterconfig.get('Mode') == 'v' or
-             milterconfig.get('Mode') == 'sv')):
+            (self.conf.get('Mode') == 'v' or
+             self.conf.get('Mode') == 'sv')):
             self.check_dkim(txt)
         if self.arresults:
             h = authres.AuthenticationResultsHeader(authserv_id=
                                                     self.AuthservID,
                                                     results=self.arresults)
-            h = fold(str(h))
-            if (milterconfig.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 2):
-                syslog.syslog(str(h))
-            name, val = str(h).split(': ', 1)
+            h = fold(codecs.encode(str(h), 'ascii'))
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 2):
+                syslog.syslog(codecs.decode(h, 'ascii'))
+            name, val = codecs.decode(h, 'ascii').split(': ', 1)
             self.addheader(name, val, 0)
         return Milter.CONTINUE
 
+    # get parent domain to be signed for if fdomain is a subdomain
+    def get_parent_domain(self, fdomain, domains):
+        for domain in domains:
+            rhs = '.'+domain
+            # compare right hand side of fdomain against .domain
+            if fdomain[-len(rhs):] == rhs:
+                # return parent domain on match
+                syslog.syslog('domain: {0}'.format(domain))
+                return domain
+        # or return the fdomain itself
+        return fdomain
+
+    def get_identities_sign(self):
+        """Determine d= and i= identiies for signature"""
+        self.domain = []
+        iequals = None
+        try:
+            self.privkeyRSA = self.conf.get('privateRSA')
+        except:
+            self.privkeyRSA = ''
+        try:
+            self.privkeyEd25519 = self.conf.get('privateEd25519')
+        except:
+            self.privkeyEd25519 = ''
+        try:
+            self.selectorRSA = self.conf.get('Selector')
+        except:
+            self.selectorRSA = ''
+        try:
+            self.selectorEd25519 = self.conf.get('SelectorEd25519')
+        except:
+            self.selectorEd25519 = ''
+        if not self.domain and self.conf.get('Domain'):
+            self.domain = self.conf.get('Domain')
+        if self.conf.get('SubDomains'):
+            self.fdomain = self.get_parent_domain(self.fdomain, self.domain)
+        if self.conf.get('SigningTable'):
+            match = False
+            for dictkey, dictvalues in self.conf.get('SigningTable').items():
+                if dictkey == '%':
+                    self.domain.append(self.fdomain)
+                    match = True
+                elif len(dictkey.split('*')) == 1:
+                    if dictkey == self.author:
+                        self.domain.append(self.fdomain)
+                        match = True
+                else:
+                    if len(dictkey.split('*')) == 2:
+                        if dictkey.split('*')[1] == self.author[-len(dictkey.split('*')[1]):]:
+                            self.domain.append(self.fdomain)
+                            match = True
+                    self.domain.append(self.fdomain)
+                try:
+                    if len(dictvalues) == 2 and match:
+                        if dictvalues[0] =='%':
+                            self.iequals = codecs.encode('@' + self.fdomain)
+                        elif dictvalues[0][1:] == self.fdomain or self.get_parent_domain(dictvalues[0][1:], self.domain) == self.fdomain:
+                            self.iequals = codecs.encode(dictvalues[0])
+                except IndexError:
+                    pass
+                if match:
+                    #TODO add KeyTable stuffs here.
+                    keytablekey = dictvalues[-1] # Last value in the SigningTable row.
+                    if self.conf.get('privateRSATable'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateRSATable')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.selectorRSA = keytabledata[1]
+                            self.privkeyRSA = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    if self.conf.get('privateEd25519Table'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateEd25519Table')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.selectorEd25519 = keytabledata[1]
+                            self.privkeyEd25519 = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    break
+
     def sign_dkim(self, txt):
-        canon = milterconfig.get('Canonicalization')
+        canon = codecs.encode(self.conf.get('Canonicalization'), 'ascii')
         canonicalize = []
-        if len(canon.split('/')) == 2:
-            canonicalize.append(canon.split('/')[0])
-            canonicalize.append(canon.split('/')[1])
+        if len(canon.split(b'/')) == 2:
+            canonicalize.append(canon.split(b'/')[0])
+            canonicalize.append(canon.split(b'/')[1])
         else:
             canonicalize.append(canon)
             canonicalize.append(canon)
-            if (milterconfig.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 1):
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog('canonicalize: {0}'.format(canonicalize))
+        sign_headers = self.conf.get('SignHeaders')
+        if not sign_headers:
+            # None or empty. DKIM explicitly tests for None.
+            sign_headers = None
         try:
-            if privateRSA:
+            if self.privkeyRSA:
                 d = dkim.DKIM(txt)
-                h = d.sign(milterconfig.get('Selector'), self.fdomain,
-                           privateRSA, canonicalize=(canonicalize[0],
-                                                     canonicalize[1]))
-                name, val = h.split(': ', 1)
-                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
-                if (milterconfig.get('Syslog') and
-                    (milterconfig.get('SyslogSuccess')
-                     or milterconfig.get('debugLevel') >= 1)):
-                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                h = d.sign(codecs.encode(self.selectorRSA, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
+                           codecs.encode(self.privkeyRSA, 'ascii'),
+                           canonicalize=(canonicalize[0], canonicalize[1]),
+                           identity=self.iequals, include_headers=sign_headers)
+                name, val = h.split(b': ', 1)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
+                if (self.conf.get('Syslog') and
+                    (self.conf.get('SyslogSuccess')
+                     or self.conf.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
-                                                  d.signature_fields.get(b's'),
-                                                  d.domain.lower()))
-            if privateEd25519:
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
+            if self.privkeyEd25519:
                 d = dkim.DKIM(txt)
-                h = d.sign(milterconfig.get('SelectorEd25519'), self.fdomain,
-                           privateEd25519, canonicalize=(canonicalize[0],
-                                                         canonicalize[1]),
-                           signature_algorithm='ed25519-sha256')
-                name, val = h.split(': ', 1)
-                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
-                if (milterconfig.get('Syslog') and
-                    (milterconfig.get('SyslogSuccess')
-                     or milterconfig.get('debugLevel') >= 1)):
-                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                h = d.sign(codecs.encode(self.selectorEd25519, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
+                           self.privkeyEd25519,
+                           canonicalize=(canonicalize[0], canonicalize[1]),
+                           identity=self.iequals, include_headers=sign_headers,
+                           signature_algorithm=b'ed25519-sha256')
+                name, val = h.split(b': ', 1)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
+                if (self.conf.get('Syslog') and
+                    (self.conf.get('SyslogSuccess')
+                     or self.conf.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
-                                                  d.signature_fields.get(b's'),
-                                                  d.domain.lower()))
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
         except dkim.DKIMException as x:
-            if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog('DKIM: {0}'.format(x))
         except Exception as x:
-            if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog("sign_dkim: {0}".format(x))
             raise
 
@@ -260,65 +359,69 @@ class dkimMilter(Milter.Base):
         res = False
         self.header_a = None
         for y in range(self.has_dkim):  # Verify _ALL_ the signatures
-            d = dkim.DKIM(txt)
+            d = dkim.DKIM(txt, minkey=self.conf.get('MinimumKeyBits'), timeout=self.conf.get('DNSTimeout'))
             try:
-                res = d.verify(idx=y)
+                dnsoverride = self.conf.get('DNSOverride')
+                if isinstance(dnsoverride, str):
+                    syslog.syslog("DNSOverride: {0}".format(dnsoverride))
+                    res = d.verify(idx=y, dnsfunc=lambda _x: dnsoverride)
+                else:
+                    res = d.verify(idx=y)
+                algo = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
                 if res:
-                    if d.signature_fields.get(b'a') == 'ed25519-sha256':
+                    if algo == 'ed25519-sha256':
                         self.dkim_comment = ('Good {0} signature'
-                                             .format(d.signature_fields
-                                                     .get(b'a')))
+                                             .format(algo))
                     else:
                         self.dkim_comment = ('Good {0} bit {1} signature'
-                                             .format(d.keysize,
-                                                     d.signature_fields
-                                                     .get(b'a')))
+                                             .format(d.keysize, algo))
                 else:
                     self.dkim_comment = ('Bad {0} bit {1} signature.'
-                                         .format(d.keysize,
-                                                 d.signature_fields.get(b'a')))
+                                         .format(d.keysize, algo))
             except dkim.DKIMException as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
                     syslog.syslog('DKIM: {0}'.format(x))
             except Exception as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
                     syslog.syslog("check_dkim: Internal program fault while verifying: {0}".format(x))
             try:
-                self.header_i = d.signature_fields.get(b'i')
+                # i= is optional and dkimpy is fine if it's not provided
+                self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
             except TypeError as x:
                 self.header_i = None
             try:
-                self.header_d = d.signature_fields.get(b'd')
-                self.header_a = d.signature_fields.get(b'a')
+                self.header_d = codecs.decode(d.signature_fields.get(b'd'), 'ascii')
+                self.header_a = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
             except Exception as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
-                    syslog.syslog("check_dkim: Internal proram fault extracting header a or d: {0}".format(x))
+                if self.conf.get('Syslog'):
+                    syslog.syslog("check_dkim: Internal program fault extracting header a or d: {0}".format(x))
                 self.header_d = None
             if not self.header_a:
                 self.header_a = 'rsa-sha256'
             if res:
-                if (milterconfig.get('Syslog') and
-                        (milterconfig.get('SyslogSuccess') or
-                         milterconfig.get('debugLevel') >= 1)):
+                if (self.conf.get('Syslog') and
+                        (self.conf.get('SyslogSuccess') or
+                         self.conf.get('debugLevel') >= 1)):
                     syslog.syslog('{0}: {1} DKIM signature verified (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
-                                                  d.signature_fields.get(b's'),
-                                                  d.domain.lower()))
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
                 self.dkim_domain = d.domain.lower()
             else:
-                if milterconfig.get('DiagnosticDirectory'):
+                if self.conf.get('DiagnosticDirectory'):
+                    tempfile.tempdir = self.conf.get('DiagnosticDirectory')
                     fd, fname = tempfile.mkstemp(".dkim")
                     with os.fdopen(fd, "w+b") as fp:
                         fp.write(txt)
-                    if milterconfig.get('Syslog'):
+                    if self.conf.get('Syslog'):
                         syslog.syslog('DKIM: Fail (saved as {0})'
                                       .format(fname))
                 else:
-                    if milterconfig.get('Syslog'):
+                    if self.conf.get('Syslog'):
                         if d.domain:
                             syslog.syslog('DKIM: Fail ({0})'
                                           .format(d.domain.lower()))
@@ -341,40 +444,48 @@ class dkimMilter(Milter.Base):
             self.header_a = None
         return
 
-
 def main():
     # Ugh, but there's no easy way around this.
     global milterconfig
-    global privateRSA
-    global privateEd25519
-    privateRSA = False
-    privateEd25519 = False
     configFile = '/usr/local/etc/dkimpy-milter.conf'
     if len(sys.argv) > 1:
-        if sys.argv[1] in ('-?', '--help', '-h'):
-            print('usage: dkimpy-milter [<configfilename>]')
+        if (sys.argv[1] in ('-?', '--help', '-h')) or len(sys.argv) == 3 or \
+               (len(sys.argv) == 4 and sys.argv[2] != '-P'):
+            print('usage: dkimpy-milter [<configfilename> [-P <pidfile>]]')
             sys.exit(1)
         configFile = sys.argv[1]
     milterconfig = config._processConfigFile(filename=configFile)
+    if len(sys.argv) == 4:
+        if sys.argv[2] == '-P':
+            # Command line PID file argument overrides config file
+            milterconfig['PidFile'] = sys.argv[3]
     if milterconfig.get('Syslog'):
         facility = eval("syslog.LOG_{0}"
                         .format(milterconfig.get('SyslogFacility').upper()))
         syslog.openlog(os.path.basename(sys.argv[0]), syslog.LOG_PID, facility)
         setExceptHook()
     pid = write_pid(milterconfig)
-    if milterconfig.get('KeyFile'):
-        privateRSA = read_keyfile(milterconfig, 'RSA')
-    if milterconfig.get('KeyFileEd25519'):
-        privateEd25519 = read_keyfile(milterconfig, 'Ed25519')
+    milterconfig = get_keys(milterconfig)
     Milter.factory = dkimMilter
     Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
     miltername = 'dkimpy-filter'
     socketname = milterconfig.get('Socket')
-    drop_privileges(milterconfig)
+    if socketname is None:
+        if int(os.environ.get('LISTEN_PID', '0')) == os.getpid():
+            lfds = os.environ.get('LISTEN_FDS')
+            if lfds is not None:
+                if lfds != '1':
+                    syslog.syslog('LISTEN_FDS is set to "{0}", but we only know how to deal with "1", ignoring it'.
+                                  format(lfds))
+                else:
+                    socketname = 'fd:3'
+        if socketname is None:
+            socketname = 'local:/var/run/dkimpy-milter/dkimpy-milter.sock'
     sys.stdout.flush()
     if milterconfig.get('Syslog'):
         syslog.syslog('dkimpy-milter starting:{0} user:{1}'
                       .format(pid, milterconfig.get('UserID')))
+    drop_privileges(milterconfig)
     Milter.runmilter(miltername, socketname, 240)
 
 if __name__ == "__main__":

dkimpy_milter/__main__.py

@@ -0,0 +1,6 @@
+#!/usr/bin/python3
+
+from dkimpy_milter import main
+
+if __name__ == "__main__":
+    main()

dkimpy_milter/config.py

@@ -31,16 +31,17 @@ import stat
 import dkim
 import socket
 import ipaddress
-from dnsplug import Session
+from .dnsplug import Session
 
 #  default values
 defaultConfigData = {
     'Syslog': 'yes',
     'SyslogFacility': 'mail',
-    'UMask': 007,
+    'UMask': 0o07,
     'Mode': 'sv',
-    'Socket': 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
-    'PidFile': '/var/run/dkimpy-milter/dkimpy-milter.pid',
+    'MinimumKeyBits': 1024,
+    'Socket': None,
+    'PidFile': None,
     'UserID': 'dkimpy-milter',
     'Canonicalization': 'relaxed/simple',
     'InternalHosts': '127.0.0.1',
@@ -48,6 +49,10 @@ defaultConfigData = {
     'DiagnosticDirectory': '',
     'MacroList': '',
     'MacroListVerify': '',
+    'DNSOverride': None,
+    'DNSTimeout': 5,
+    'SubDomains': False,
+    'SigningTable': None,
     'debugLevel': 0  # Undocumented config item for developer use
     }
 
@@ -84,14 +89,14 @@ class HostsDataset(object):
                 self.item = item[1:]
                 self.negative = True
             try:
-                self.item = ipaddress.ip_address(unicode(self.item, "utf-8"))
+                self.item = ipaddress.ip_address(str(self.item, "utf-8"))
                 if isinstance(self.item, ipaddress.IPv4Address):
                     self.isipv4 = True
                 elif isinstance(self.item, ipaddress.IPv6Address):
                     self.isipv6 = True
             except ValueError as e:
                 try:
-                    self.item = ipaddress.ip_network(unicode
+                    self.item = ipaddress.ip_network(str
                                                      (self.item, "utf-8"),
                                                      strict=False)
                     if isinstance(self.item, ipaddress.IPv4Network):
@@ -109,7 +114,7 @@ class HostsDataset(object):
 
     def match(self, connectip):
         '''Check if the connect IP is part of the dataset'''
-        source = ipaddress.ip_address(unicode(connectip, "utf-8"))
+        source = ipaddress.ip_address(str(connectip, "utf-8"))
         for item in self.dataset:
             if item.isdomain or item.ishostname:
                 result = self.matchname(source)   # Match host/domains first
@@ -154,18 +159,18 @@ class HostsDataset(object):
         '''Get validated PTR name of IP address'''
         results = []
         s = Session()
-        ptrnames = s.dns(source.reverse_pointer, 'PTR')
+        ptrnames = s.dns(source.reverse_pointer, 'PTR', timeout=self.conf.get('DNSTimeout'))
         for name in ptrnames:
             if isinstance(source, ipaddress.IPv4Address):
                 ips = s.dns(name, 'A')
                 for ip in ips:
-                    ip = ipaddress.IPv4Address(unicode(ip, 'UTF-8'))
+                    ip = ipaddress.IPv4Address(str(ip, 'UTF-8'))
                     if ip == source:
                         results.append(name)
             if isinstance(source, ipaddress.IPv6Address):
                 ips = s.dns(name, 'AAAA')
                 for ip in ips:
-                    ip = ipaddress.IPv6Address(unicode(ip, 'UTF-8'))
+                    ip = ipaddress.IPv6Address(str(ip, 'UTF-8'))
                 if ip == source:
                     results.append(name)
         return results
@@ -224,13 +229,13 @@ def _processConfigFile(filename=None, configdata=None, useSyslog=1,
     '''Load the specified config file, exit and log errors if it fails,
     otherwise return a config dictionary.'''
 
-    import config
+    from . import config
     if configdata is None:
         configdata = config.defaultConfigData
     if filename is not None:
         try:
             _readConfigFile(filename, configdata)
-        except Exception, e:
+        except Exception as e:
             raise
             if useSyslog:
                 syslog.syslog(e.args[0])
@@ -258,10 +263,10 @@ def _make_authserv_id(as_id):
         as_id = socket.gethostname()
     return as_id
 
-
 def _dataset_to_list(dataset):
     """Convert a dataset (as defined in dkimpymilter.8) and return a python
-       list of values."""
+       list of values.  For multiline datasets like KeyTable and SigningTable a
+       key : values dictionary is returned"""
     if not isinstance(dataset, str):
         # If it was a csl with more than one value, it's already a list, we
         # only need to remove the name from the first value.
@@ -271,23 +276,34 @@ def _dataset_to_list(dataset):
             dataset[dataset.index(item)] = item.strip().strip(',')
         return dataset
     elif isinstance(dataset, str):
-        if dataset[0] == '/' or dataset[:5] == 'file:':
-            # This is a flat file dataset
+        if dataset[0] == '/' or dataset[:5] == 'file:' or dataset[:7] == 'refile:':
+            # This is a flat file dataset, which are key value:value stores
             ds = []
-            if dataset[0] == '/':
+            dsd = {}
+            if dataset[0] == '/' or dataset[:2] == './' or dataset[:3] == '../':
                 dsname = dataset
-            if dataset[:5] == 'file:':
+            elif dataset[:5] == 'file:':
                 dsname = dataset[5:]
+            elif dataset[:7] == 'refile:':
+                dsname = dataset[7:]
             dsf = open(dsname, 'r')
             for line in dsf.readlines():
                 if line[0] != '#':
-                    if len(line.split(':')) == 1:
-                        ds.append(line.strip())
-                    else:
-                        for element in line.split(':'):
-                            ds.append(element.strip().strip(':'))
+                    if len(line.split()) == 1:
+                        if len(line.split(':')) == 1:
+                            ds.append(line.strip())
+                        else:
+                            for element in line.split(':'):
+                                ds.append(element.strip().strip(':'))
+                    elif len(line.split()) == 2: # key value:value:value
+                        key, values = line.split()
+                        values = values.split(':')
+                        dsd.update({key:values})
             dsf.close()
-            return ds
+            if ds:
+                return ds
+            elif dsd:
+                return dsd
         # If it's a str and csl, it has one value and we return a list
         if dataset[:4] == 'csl:':
             return [dataset[4:].strip().strip(',')]
@@ -308,7 +324,9 @@ def _readConfigFile(path, configData=None, configGlobal={}):
     dictionary of name/value pairs based on configData and the values
     read from path.'''
 
-    debugLevel = configGlobal.get('debugLevel', 0)
+    # No config file data is available yet, so to debug _readConfigFile, set
+    # the value here.
+    debugLevel = 0
     if debugLevel >= 5:
         syslog.syslog('readConfigFile: Loading "%s"' % path)
     if configData is None:
@@ -320,27 +338,35 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         'SyslogSuccess': 'bool',
         'UMask': 'int',
         'Mode': 'str',
+        'MinimumKeyBits': 'int',
         'Socket': 'str',
         'PidFile': 'str',
         'UserID': 'str',
         'Domain': 'dataset',
+        'SubDomains': 'bool',
         'KeyFile': 'str',
+        'KeyTable': 'dataset',
         'KeyFileEd25519': 'str',
+        'KeyTableEd25519': 'dataset',
         'Selector': 'str',
         'SelectorEd25519': 'str',
+        'SigningTable': 'dataset',
         'Canonicalization': 'str',
         'InternalHosts': 'dataset',
         'IntHosts': 'bool',
         'DiagnosticDirectory': 'str',
         'MacroList': 'dataset',
         'MacroListVerify': 'dataset',
-        'debugLevel': 'int'
+        'DNSOverride': 'str',
+        'DNSTimeout': 'int',
+        'debugLevel': 'int',
+        'SignHeaders': 'dataset'
         }
 
     #  check to see if it's a file
     try:
         mode = os.stat(path)[0]
-    except OSError, e:
+    except OSError as e:
         syslog.syslog(syslog.LOG_ERR, 'ERROR stating "%s": %s'
                       % (path, e.strerror))
         return(configData)
@@ -376,9 +402,15 @@ def _readConfigFile(path, configData=None, configGlobal={}):
             value = data[1:]
 
         #  check validity of name
-        conversion = nameConversion.get(name)
+        try:
+            conversion = nameConversion.get(name)
+        except TypeError:
+            name = name[0]
+            syslog.syslog('Config item "%s" does not provide a value in file "%s"'
+                          % (name, path))
+            conversion = None
         if conversion is None:
-            syslog.syslog('ERROR: Unknown name "%s" in file "%s"'
+            syslog.syslog('ERROR: Unknown name or name missing value "%s" in file "%s"'
                           % (name, path))
             continue
 
@@ -388,8 +420,15 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         if conversion == 'bool':
             configData[name] = _find_boolean(value)
         elif conversion == 'str':
-            configData[name] = str(value)
+            if isinstance(value, list):
+                configData[name] = line.split(None, 1)[1]
+            else:
+                configData[name] = str(value)
         elif conversion == 'int':
+            if name == 'MinimumKeyBits':
+                if int(value) == 0:
+                    # Odd inheritence from OpenDKIM where value of 0 means use default.
+                    value = configData.get(name)
             configData[name] = int(value)
         elif conversion == 'dataset':
             configData[name] = _dataset_to_list(value)
@@ -399,7 +438,7 @@ def _readConfigFile(path, configData=None, configGlobal={}):
             configData[name] = conversion(value)
     fp.close()
     try:
-        configData['AuthservID'] = _make_authserv_id(configData['AuthservID'])
+        configData['AuthservID'] = _make_authserv_id(configData.get('AuthservID', 'HOSTNAME'))
         configData['IntHosts'] = HostsDataset(configData['InternalHosts'])
     except:
         pass

dkimpy_milter/dnsplug.py

@@ -84,11 +84,11 @@ class Session(object):
             raise DNSError('Length of CNAME chain exceeds %d' % MAX_CNAME)
         cnames[name] = cname
         if cname in cnames:
-            raise DNSError, 'CNAME loop'
+            raise DNSError('CNAME loop')
         result = self.dns(cname, qtype, cnames=cnames)
     return result
 
-def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
+def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=5):
     try:
 	# FIXME: To be thread safe, we create a fresh DnsRequest with
 	# each call.  It would be more efficient to reuse
@@ -103,22 +103,22 @@ def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
         #
         if resp.header['tc'] == True:
           if not tcpfallback:
-              raise DNS.DNSError, 'DNS: Truncated UDP Reply, SPF records should fit in a UDP packet'
+              raise DNS.DNSError('DNS: Truncated UDP Reply, SPF records should fit in a UDP packet')
           try:
               req = DNS.DnsRequest(name, qtype=qtype, protocol='tcp',
                         timeout=timeout)
               resp = req.req()
-          except DNS.DNSError, x:
-              raise DNS.DNSError, 'TCP Fallback error: ' + str(x)
+          except DNS.DNSError as x:
+              raise DNS.DNSError('TCP Fallback error: ' + str(x))
         return [((a['name'], a['typename']), a['data']) for a in resp.answers]
-    except IOError, x:
-        raise DNS.DNSError, 'DNS: ' + str(x)
+    except IOError as x:
+        raise DNS.DNSError('DNS: ' + str(x))
 
-def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=30):
+def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=5):
   retVal = []
   try:
     # FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
-    answers = dns.resolver.query(name, qtype)
+    answers = dns.resolver.query(name, qtype, raise_on_no_answer=False, lifetime=timeout)
     for rdata in answers:
       if qtype == 'A' or qtype == 'AAAA':
         retVal.append(((name, qtype), rdata.address))
@@ -164,5 +164,5 @@ if __name__ == '__main__':
   import sys
   s = Session()
   for n,t in zip(*[iter(sys.argv[1:])]*2):
-    print n,t
-    print s.dns(n,t)
+    print(n,t)
+    print(s.dns(n,t))

dkimpy_milter/util.py

@@ -115,44 +115,43 @@ def write_pid(milterconfig):
     """Write PID in pidfile.  Will not overwrite an existing file."""
     import os
     import syslog
-    if not os.path.isfile(milterconfig.get('PidFile')):
+    pidfile = milterconfig.get('PidFile')
+    if pidfile is None:
+        return
+    if not os.path.isfile(pidfile):
         pid = str(os.getpid())
         try:
-            f = open(milterconfig.get('PidFile'), 'w')
+            f = open(pidfile, 'w')
         except IOError as e:
             if str(e)[:35] == '[Errno 2] No such file or directory':
-                piddir = milterconfig.get('PidFile').rsplit('/', 1)[0]
+                piddir = pidfile.rsplit('/', 1)[0]
                 os.mkdir(piddir)
                 user, group = user_group(milterconfig.get('UserID'))
                 os.chown(piddir, user, group)
-                f = open(milterconfig.get('PidFile'), 'w')
+                f = open(pidfile, 'w')
                 if milterconfig.get('Syslog'):
                     syslog.syslog('PID dir created: {0}'.format(piddir))
             else:
                 if milterconfig.get('Syslog'):
                     syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'
-                                  .format(milterconfig.get('PidFile'), e))
+                                  .format(pidfile, e))
                 raise
         f.write(pid)
         f.close()
         user, group = user_group(milterconfig.get('UserID'))
-        os.chown(milterconfig.get('PidFile'), user, group)
+        os.chown(pidfile, user, group)
     else:
         if milterconfig.get('Syslog'):
             syslog.syslog('Unable to write pidfle {0}.  File exists.'
-                          .format(milterconfig.get('PidFile')))
+                          .format(pidfile))
         raise RuntimeError('Unable to write pidfle {0}.  File exists.'
-                           .format(milterconfig.get('PidFile')))
+                           .format(pidfile))
     return pid
 
 
-def read_keyfile(milterconfig, keytype):
+def read_keyfile(keyfile, milterconfig):
     """Read private key from file."""
     import syslog
-    if keytype == "RSA":
-        keyfile = milterconfig.get('KeyFile')
-    if keytype == "Ed25519":
-        keyfile = milterconfig.get('KeyFileEd25519')
     try:
         f = open(keyfile, 'r')
         keylist = f.readlines()
@@ -166,3 +165,37 @@ def read_keyfile(milterconfig, keytype):
     for line in keylist:
         key += line
     return key
+
+def read_keytable(tabledict, milterconfig):
+    """Read keytables into in memory configuration data so all keys are read
+    before priviledges are dropped.
+    When loaded, tabeldict is a dict:
+        {searchkey: [donamin, selector, key]}
+    If key is a file (startswith('/'), then the key is returned in its place."""
+    import dkim
+    import syslog
+    for dictkey, values in tabledict.items():
+        if values[-1][:1] == '/' or values[-1][:2] == './' or values[-1][:3] == '../':
+            key = read_keyfile(values[-1], milterconfig)
+        tabledict[dictkey] = [values[0], values[1], key]
+    return tabledict
+
+def get_keys(milterconfig):
+    """Read keys (table or file) into memory before dropping priviledges"""
+    milterconfig['privateRSA'] = False
+    milterconfig['privateRSATable'] = False
+    milterconfig['privateEd25519'] = False
+    milterconfig['privateEd25519Table'] = False
+    if milterconfig.get('KeyTable'):
+        milterconfig['privateRSATable'] = read_keytable(milterconfig.get('KeyTable'),
+                                                        milterconfig)
+    elif milterconfig.get('KeyFile'):
+        milterconfig['privateRSA'] = read_keyfile(milterconfig.get('KeyFile'),
+                                                  milterconfig)
+    if milterconfig.get('KeyTableEd25519'):
+        milterconfig['privateEd25519Table'] = read_keytable(milterconfig.get('KeyTableEd25519'),
+                                                            milterconfig)
+    elif milterconfig.get('KeyFileEd25519'):
+        milterconfig['privateEd25519'] = read_keyfile(milterconfig.get('KeyFileEd25519'),
+                                                      milterconfig)
+    return milterconfig

etc/dkimpy-milter.conf

@@ -1,6 +1,5 @@
 # This is a basic configuration that can easily be adapted to suit a standard
-# installation. For more advanced options, see dkimpy-milter.conf(5) and/or
-# /usr/share/doc/dkimpy-milter/examples/opendkim.conf.sample.
+# installation. For more advanced options, see dkimpy-milter.conf(5).
 
 # Log to syslog
 Syslog			yes
@@ -9,18 +8,16 @@ Syslog			yes
 # privileged user (e.g. Postfix)
 UMask			007
 
-# Sign for example.com with key in /etc/dkimkeys/dkim.key using
+# Sign for example.com with key in /usr/local/etc/dkimkeys/dkim.key using
 # selector '2007' (e.g. 2007._domainkey.example.com)
 #Domain			example.com
-#KeyFile			/etc/mail/dkim.key
+#KeyFile		/usr/local/etc/mail/dkim.key
 #Selector		default
 
 # Commonly-used options; the commented-out versions show the defaults.
 #Canonicalization	relaxed/simple
 #Mode			sv
 
-# Socket local:/var/run/dkimpy-milter/dkimpy-milter.sock
-#
 # ##  Socket socketspec
 # ##
 # ##  Names the socket where this filter should listen for milter connections
@@ -30,15 +27,17 @@ UMask			007
 # ##  inet:port                   to listen on all interfaces
 # ##  local:/path/to/socket       to listen on a UNIX domain socket
 #
-Socket			inet:8892@localhost
+#Socket local:/run/dkimpy-milter/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
 
 ##  PidFile filename
-###      default /var/run/dkimpy-milter/dkimpy-milter.pid
+###      default /run/dkimpy-milter/dkimpy-milter.pid
 ###
 ###  Name of the file where the filter should write its pid before beginning
 ###  normal operations.
 #
-PidFile			/var/run/dkimpy-milter/dkimpy-milter.pid
+PidFile			/run/dkimpy-milter/dkimpy-milter.pid
 
 ##  Userid userid
 ###      default dkimpy-milter

etc/dkimpy-milter.conf.in

@@ -0,0 +1,48 @@
+# This is a basic configuration that can easily be adapted to suit a standard
+# installation. For more advanced options, see dkimpy-milter.conf(5).
+
+# Log to syslog
+Syslog			yes
+
+# Required to use local socket with MTAs that access the socket as a non-
+# privileged user (e.g. Postfix)
+UMask			007
+
+# Sign for example.com with key in @SYSCONFDIR@/dkimkeys/dkim.key using
+# selector '2007' (e.g. 2007._domainkey.example.com)
+#Domain			example.com
+#KeyFile		@SYSCONFDIR@/mail/dkim.key
+#Selector		default
+
+# Commonly-used options; the commented-out versions show the defaults.
+#Canonicalization	relaxed/simple
+#Mode			sv
+
+# ##  Socket socketspec
+# ##
+# ##  Names the socket where this filter should listen for milter connections
+# ##  from the MTA.  Required.  Should be in one of these forms:
+# ##
+# ##  inet:port@address           to listen on a specific interface
+# ##  inet:port                   to listen on all interfaces
+# ##  local:/path/to/socket       to listen on a UNIX domain socket
+#
+#Socket local:@RUNSTATEDIR@/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
+
+##  PidFile filename
+###      default /run/dkimpy-milter/dkimpy-milter.pid
+###
+###  Name of the file where the filter should write its pid before beginning
+###  normal operations.
+#
+PidFile			@RUNSTATEDIR@/dkimpy-milter.pid
+
+##  Userid userid
+###      default dkimpy-milter
+###
+###  Change to user "userid" before starting normal operation?  May include
+###  a group ID as well, separated from the userid by a colon.
+#
+UserID			dkimpy-milter

man/dkimpy-milter.8

@@ -170,22 +170,22 @@ are ignored, and the hash ("#") character denotes the start of a comment.
 If a value contains multiple entries, the entries should be separated by
 colons.
 .TP
-.I c)
-If the string begins with "db:" and the program was compiled with
-Sleepycat DB support, then the remainder of the string is presumed to
-identify a Sleepycat database containing keys and corresponding values.
-These may be used only to test for membership in the data set, or for
-storing keys and corresponding values.  If a value contains multiple entries,
-the entries should be separated by colons. [Not implemented yet]
-.TP
-.I h)
-If the string contains none of these prefixes but ends with ".db", it
-is presumed to be a Sleepycat DB as described above (if support for same
-is compiled in).  [Not implemented yet]
+.I b)
+If the string begins with "refile:", then the remainder of the string is
+presumed to specify a file that contains a set of patterns, one per line,
+and their associated values.  The pattern is taken as the start of the line
+to the first whitespace, and the portion after that whitespace is taken as
+the value to be used when that pattern is matched.  Patterns are simple
+wildcard patterns, matching all text except that the asterisk ("*") character
+is considered a wildcard.  If a value contains multiple entries, the entries
+should be separated by colons.
 .TP
 .I i)
 If the string contains none of these prefixes but starts with a slash ("/")
-character, it is presumed to be a flat file as described above.
+character, or "./" or "../", it is presumed to be a flat file as described
+above. Note: In OpenDKIM "./" and "../" only apply to KeyTable, but for
+dkimpy-milter it is generally applicable and KeyTable specification is not
+a special case.
 .TP
 .I j)
 If the string begins with "csl:", the string is treated as a comma-separated
@@ -203,9 +203,10 @@ pairs as described above.
 .SH OPTIONS
 .TP
 See
-.I dkimpy-milter.conf(5)
-information about available options.  Unlike OpenDKIM, dkimpy-milter does not
-support command line option switches.
+.I dkimpy-milter.conf (5)
+for information about available options.  Unlike OpenDKIM, with the exception of
+\-P for the pidfile and specifying the configuration file to use,
+dkimpy-milter does not support command line option switches.
 
 When signing a message, a
 .I DKIM-Signature:

man/dkimpy-milter.conf.5

@@ -127,11 +127,11 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.TH dkimpy-milter.conf 5 "2018-02-12"
+.TH dkimpy-milter.conf 5 "2019-04-25"
 .SH "NAME"
 dkimpy-milter \- Python milter for DKIM signing and validation
 .SH "VERSION"
-0\.9\.2
+1\.2\.0
 
 .SH "DESCRIPTION"
 .I dkimpy-milter(8)
@@ -152,13 +152,14 @@ the value is processed.  For positive values, the following are accepted:
 "F", "f", "N", "n", "0".
 
 The provided setup.py installs this configuration file in /etc or
-/usr/local/etc.
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
 
 Command line invocation of parameters as is done by OpenDKIM is not supported.
 
 .SH "USAGE"
 Usage:
-  dkimpy-milter [/etc/dkimpy-milter.conf]
+  dkimpy-milter [/usr/local/etc/dkimpy-milter/dkimpy-milter.conf]
 
 .SH "OTHER DOCUMENTATION"
 This documentation assumes you have read Postfix's README_FILES/MILTER_README
@@ -210,9 +211,11 @@ header and the second to the body.
 .TP
 .I DiagnosticDirectory (string)
 Directory into which to write diagnostic reports when message verification
-fails.  If not set (the default), these files are not generated.  [Unlike
-OpenDKIM, this applies to all messages, not just  on messages bearing a "z=" tag
-because dkimpy does not yet support "z=".]
+fails.  If not set (the default), these files are not generated.  The
+directory must exist, dkimpy-milter will not create it and an error will be
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
 
 .TP
 .I Domain (dataset)
@@ -222,11 +225,13 @@ domains will be verified rather than being signed.
 This parameter is not required if a
 .I SigningTable
 is in use; in that case, the list of signed domains is implied by the
-lines in that file. [SigningTable NOT IMPLEMENTED]
+lines in that file.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+or
+.I KeyTableD25119
+is defined.
 
 .TP
 .I InternalHosts (dataset)
@@ -244,7 +249,7 @@ address explicitly. [PeerList NOT IMPLEMENTED]
 Gives the location of a PEM-formatted private key to be used for RSA signing
 all messages.  Ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
 .I KeyFileEd25519 (string)
@@ -252,7 +257,19 @@ Gives the location of a Ed25519 private key to be used for Ed25519 signing
 all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
 generation (as used in dkimpy).  Ignored if a 
 .I KeyTableEd25519
-is defined.  [KeyTableEd25519 NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
 
 .TP
 .I MacroList (dataset)
@@ -309,7 +326,43 @@ When signing mode is enabled, one of the following combinations must also
 be set:
 (a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
 (b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
-[fooTable options NOT IMPLEMENTED]
+
+TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
+
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
 
 .TP
 .I PeerList (dataset)
@@ -331,7 +384,7 @@ will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
 .TP
 .I PidFile (string)
 Specifies the path to a file that should be created at process start
-containing the process ID.
+containing the process ID.  If not specified, no such file will be created.
 
 .TP
 .I Selector (string)
@@ -345,7 +398,7 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
 .I SelectorEd25519 (string)
@@ -359,7 +412,33 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTableEd25519
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I SignHeaders (dataset)
+Specifies the set of header fields that should be included when generating
+signatures.  If the list omits any header field that is mandated by the DKIM
+specification, those fields are implicitly added.  By default, those fields
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
+will be signed by the filter.  See the
+.I OmitHeaders
+configuration option for more information about the format and interpretation
+of this field.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select one or more signing identities to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
 
 .TP
 .I Socket (string)
@@ -384,6 +463,12 @@ is not given as either a hostname or an IP address, the socket will be
 listening on all interfaces.  A literal IP address must be enclosed in
 square brackets.  This option is mandatory in the configuration file.
 
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
 .TP
 .I Syslog (Boolean)
 Log via calls to
@@ -432,11 +517,32 @@ unless an alternate
 .I group
 is specified.
 
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
+Default location of this file.
+
 .SH "AUTHORS"
 \ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
-It is based on dkimpy-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
 Copyright (c) 2013-2015 Stuart D. Gathman
-Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>.  
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
 .PP
 This man-page was created by Scott Kitterman <scott@kitterman.com>.
 
@@ -450,4 +556,4 @@ See LICENSE.
 
 Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
 of the package.
-Copyright (c) 2018, Scott Kitterman <scott@kitterman.com>
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

man/dkimpy-milter.conf.5.in

@@ -0,0 +1,559 @@
+\"
+.\" Standard preamble:
+.\" ========================================================================
+.de Sh \" Subsection heading
+.br
+.if t .Sp
+.ne 5
+.PP
+\fB\\$1\fR
+.PP
+..
+.de Sp \" Vertical space (when we can't use .PP)
+.if t .sp .5v
+.if n .sp
+..
+.de Vb \" Begin verbatim text
+.ft CW
+.nf
+.ne \\$1
+..
+.de Ve \" End verbatim text
+.ft R
+.fi
+..
+.\" Set up some character translations and predefined strings.  \*(-- will
+.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
+.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
+.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
+.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
+.\" nothing in troff, for use with C<>.
+.tr \(*W-
+.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
+.ie n \{\
+.    ds -- \(*W-
+.    ds PI pi
+.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
+.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
+.    ds L" ""
+.    ds R" ""
+.    ds C` ""
+.    ds C' ""
+'br\}
+.el\{\
+.    ds -- \|\(em\|
+.    ds PI \(*p
+.    ds L" ``
+.    ds R" ''
+'br\}
+.\"
+.\" If the F register is turned on, we'll generate index entries on stderr for
+.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index
+.\" entries marked with X<> in POD.  Of course, you'll have to process the
+.\" output yourself in some meaningful fashion.
+.if \nF \{\
+.    de IX
+.    tm Index:\\$1\t\\n%\t"\\$2"
+..
+.    nr % 0
+.    rr F
+.\}
+.\"
+.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
+.\" way too many mistakes in technical documents.
+.hy 0
+.if n .na
+.\"
+.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
+.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
+.    \" fudge factors for nroff and troff
+.if n \{\
+.    ds #H 0
+.    ds #V .8m
+.    ds #F .3m
+.    ds #[ \f1
+.    ds #] \fP
+.\}
+.if t \{\
+.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
+.    ds #V .6m
+.    ds #F 0
+.    ds #[ \&
+.    ds #] \&
+.\}
+.    \" simple accents for nroff and troff
+.if n \{\
+.    ds ' \&
+.    ds ` \&
+.    ds ^ \&
+.    ds , \&
+.    ds ~ ~
+.    ds /
+.\}
+.if t \{\
+.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
+.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
+.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
+.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
+.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
+.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
+.\}
+.    \" troff and (daisy-wheel) nroff accents
+.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
+.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
+.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
+.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
+.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
+.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
+.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
+.ds ae a\h'-(\w'a'u*4/10)'e
+.ds Ae A\h'-(\w'A'u*4/10)'E
+.    \" corrections for vroff
+.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
+.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
+.    \" for low resolution devices (crt and lpr)
+.if \n(.H>23 .if \n(.V>19 \
+\{\
+.    ds : e
+.    ds 8 ss
+.    ds o a
+.    ds d- d\h'-1'\(ga
+.    ds D- D\h'-1'\(hy
+.    ds th \o'bp'
+.    ds Th \o'LP'
+.    ds ae ae
+.    ds Ae AE
+.\}
+.rm #[ #] #H #V #F C
+.\" ========================================================================
+.\"
+.TH dkimpy-milter.conf 5 "2019-04-25"
+.SH "NAME"
+dkimpy-milter \- Python milter for DKIM signing and validation
+.SH "VERSION"
+1\.2\.0
+
+.SH "DESCRIPTION"
+.I dkimpy-milter(8)
+implements the
+.B DKIM
+specification for signing and verifying e-mail messages on a per-domain
+basis.  This file is its configuration file.
+
+Blank lines are ignored.  Lines containing a hash ("#") character are
+truncated at the hash character to allow for comments in the file.
+
+Other content should be the name of a parameter, followed by white space,
+followed by the value of that parameter, each on a separate line.
+
+For parameters that are Boolean in nature, only the first byte of
+the value is processed.  For positive values, the following are accepted:
+"T", "t", "Y", "y", "1".  For negative values, the following are accepted:
+"F", "f", "N", "n", "0".
+
+The provided setup.py installs this configuration file in /etc or
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
+
+Command line invocation of parameters as is done by OpenDKIM is not supported.
+
+.SH "USAGE"
+Usage:
+  dkimpy-milter [@CONFDIR@/dkimpy-milter.conf]
+
+.SH "OTHER DOCUMENTATION"
+This documentation assumes you have read Postfix's README_FILES/MILTER_README
+(or Sendmail equivalent) and are generally familiar with Domain Keys Identified
+Mail (DKIM).  See RFC 6376 for details.
+
+.SH "SYNOPSIS"
+
+dkimpy-milter operates with a default installed configuration file and 
+set of default configuration options that are used if the configuration file
+cannot be found.  These options can be changed by changing the installed 
+configuration files.  For users transitioning from OpenDKIM, OpenDKIM config
+files can be used directly.  Not all OpenDKIM options are supported.  If an
+unsupported option from OpenDKIM is specified, an error will be raised.
+
+.SH "DESCRIPTION"
+
+Configuration options are described here and in the configuration file 
+provided with the package.  The provided setup.py installs this configuration 
+file in /etc or /usr/local/etc.
+
+.SH "OPTIONS"
+
+.TP
+.I AuthservID (string)
+Sets the "authserv-id" to use when generating the Authentication-Results:
+header field after verifying a message.  The default is to use the name of
+the MTA processing the message.  If the string "HOSTNAME" is provided, the
+name of the host running the filter (as returned by the
+.I gethostname(3)
+function) will be used.
+
+.TP
+.I Canonicalization (string)
+Selects the canonicalization method(s) to be used when signing messages.
+When verifying, the message's DKIM-Signature: header field specifies
+the canonicalization method.  The recognized values are
+.I relaxed
+and
+.I simple
+as defined by the DKIM specification.  The default is
+.I relaxed
+/
+.I simple.
+The value may include two different canonicalizations separated by a
+slash ("/") character, in which case the first will be applied to the
+header and the second to the body.
+
+.TP
+.I DiagnosticDirectory (string)
+Directory into which to write diagnostic reports when message verification
+fails.  If not set (the default), these files are not generated.  The
+directory must exist, dkimpy-milter will not create it and an error will be
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
+
+.TP
+.I Domain (dataset)
+A set of domains whose mail should be signed by this filter.  Mail from other
+domains will be verified rather than being signed.
+
+This parameter is not required if a
+.I SigningTable
+is in use; in that case, the list of signed domains is implied by the
+lines in that file.
+
+This parameter is ignored if a
+.I KeyTable
+or
+.I KeyTableD25119
+is defined.
+
+.TP
+.I InternalHosts (dataset)
+Identifies a set internal hosts whose mail should be signed rather
+than verified.  Entries in this data set follow the same form as those of
+the
+.I PeerList
+option below.  If not specified, the default of "127.0.0.1" is applied.
+Naturally, providing a value here overrides the default, so if mail from
+127.0.0.1 should be signed, the list provided here should include that
+address explicitly. [PeerList NOT IMPLEMENTED]
+
+.TP
+.I KeyFile (string)
+Gives the location of a PEM-formatted private key to be used for RSA signing
+all messages.  Ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I KeyFileEd25519 (string)
+Gives the location of a Ed25519 private key to be used for Ed25519 signing
+all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
+generation (as used in dkimpy).  Ignored if a 
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I MacroList (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be a
+local user and therefore whether or not the message should be signed.  If
+a
+.I value
+is specified matching a macro name in the data set, the value of the macro
+must match a value specified (matching is case-sensitive), otherwise the
+macro must be defined but may contain any value.  The set is empty by
+default, meaning macros are not considered when making the sign-verify
+decision.  The general format of the value is
+.I value1[|value2[|...]];
+if one or more value is defined then the macro must be set to one of the
+listed values, otherwise the macro must be set but can contain any
+value.
+
+In order for the macro and its value to be available to the filter for
+checking, the MTA must send it during the protocol exchange.  This is either
+accomplished via manual configuration of the MTA to send the desired macros
+or, for MTA/filter combinations that support the feature, the filter can
+request those macros that are of interest.  The latter is a feature negotiated
+at the time the filter receives a connection from the MTA and its availability
+depends upon the version of milter used to compile the filter and the version
+of the MTA making the connection.
+
+.TP
+.I MacroListVerify (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be an
+external source and therefore whether or not the message should be signed.
+Entries in this data set follow the same form as those of the
+.I MacroList
+option above.  [this option is not inhereted from OpenDKIM]  
+
+.TP
+.I Mode (string)
+Selects operating modes.  The string is a concatenation of characters that
+indicate which mode(s) of operation are desired.  Valid modes are
+.I s
+(signer) and
+.I v
+(verifier).  The default is
+.I sv
+except in test mode (see the
+.I opendkim(8)
+man page)
+in which case the default is
+.I v.
+When signing mode is enabled, one of the following combinations must also
+be set:
+(a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
+(b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
+
+TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
+
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
+
+.TP
+.I PeerList (dataset)
+Identifies a set of "peers" that identifies clients whose connections
+should be accepted without processing by this filter.  The set
+should contain on each line a hostname, domain name (e.g. ".example.com"),
+IP address, an IPv6 address (including an IPv4 mapped address), or a
+CIDR-style IP specification (e.g. "192.168.1.0/24").  An entry beginning
+with a bang ("!") character means "not", allowing exclusions of specific
+hosts that are otherwise members of larger sets.  Host and domain names are 
+matched first, then the IP or IPv6 address depending on the connection 
+type.  More precise entries are preferred over less precise ones, i.e. 
+"192.168.1.1" will match before "!192.168.1.0/24".  The text form of IPv6 
+addresses will be forced to lowercase when queried (RFC5952), so the contents
+of this data set should also use lowercase.  The IP address portion of an
+entry may optionally contain square brackets; both forms (with and without)
+will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
+
+.TP
+.I PidFile (string)
+Specifies the path to a file that should be created at process start
+containing the process ID.  If not specified, no such file will be created.
+
+.TP
+.I Selector (string)
+Defines the name of the selector to be used when signing messages using RSA.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I SelectorEd25519 (string)
+Defines the name of the selector to be used when signing messages using Ed25519.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I SignHeaders (dataset)
+Specifies the set of header fields that should be included when generating
+signatures.  If the list omits any header field that is mandated by the DKIM
+specification, those fields are implicitly added.  By default, those fields
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
+will be signed by the filter.  See the
+.I OmitHeaders
+configuration option for more information about the format and interpretation
+of this field.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select one or more signing identities to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I Socket (string)
+Specifies the socket that should be established by the filter to receive
+connections from
+.I postfix(1)
+in order to provide service.
+.I socketspec
+is in one of two forms:
+.I local:path,
+which creates a UNIX domain socket at the specified
+.I path,
+or
+.I inet:port[@host]
+or
+.I inet6:port[@host]
+which creates a TCP socket on the specified
+.I port
+and in the specified protocol family.  If the
+.I host
+is not given as either a hostname or an IP address, the socket will be
+listening on all interfaces.  A literal IP address must be enclosed in
+square brackets.  This option is mandatory in the configuration file.
+
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
+.TP
+.I Syslog (Boolean)
+Log via calls to
+.I syslog(3)
+any interesting activity.
+
+.TP
+.I SyslogFacility (string)
+Log via calls to
+.I syslog(3)
+using the named facility.  The facility names are the same as the ones
+allowed in
+.I syslog.conf(5).
+The default is "mail".
+
+.TP
+.I SyslogSuccess (Boolean)
+Log via calls to
+.I syslog(3)
+additional entries indicating successful signing or verification of
+messages.
+
+.TP
+.I UMask (integer)
+Requests a specific permissions mask to be used for file creation.
+This only really applies to creation of the socket when
+.I Socket
+specifies a UNIX domain socket, and to the
+.I PidFile
+(if any); temporary files are created by the
+.I mkstemp(3)
+function that enforces a specific file mode on creation regardless
+of the process umask.  See
+.I umask(2)
+for more information.
+
+.TP
+.I UserID (string)
+Attempts to become the specified userid before starting operations.
+The value is of the form
+.I userid[:group].
+The process will be assigned all of the groups and primary group ID of
+the named
+.I userid
+unless an alternate
+.I group
+is specified.
+
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I @CONFDIR@/dkimpy-milter.conf
+Default location of this file.
+
+.SH "AUTHORS"
+\ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+Copyright (c) 2013-2015 Stuart D. Gathman
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
+.PP
+This man-page was created by Scott Kitterman <scott@kitterman.com>.
+
+.SH COPYRIGHT
+Configuration items derived from OpenDKIM 2.11.0 opendkim.conf.5.in:
+Copyright (c) 2007, 2008, Sendmail, Inc. and its suppliers.  All rights
+reserved.  See LICENSE.Sendmail.
+
+Copyright (c) 2009-2015, The Trusted Domain Project.  All rights reserved.
+See LICENSE.
+
+Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
+of the package.
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

setup.py

@@ -1,7 +1,7 @@
-#! /usr/bin/python
+#! /usr/bin/python3
 # dkimpy-milter: A DKIM signing/verification Milter application
 # Author: Scott Kitterman <scott@kitterman.com>
-# Copyright 2018 Scott Kitterman
+# Copyright 2018,2019 Scott Kitterman
 """    This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; either version 2 of the License, or
@@ -17,24 +17,85 @@
     51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA."""
 
 from setuptools import setup
+import distutils.cmd
+import distutils.log
+import sys
 import os
+import subprocess
 
 description = "Domain Keys Identified Mail (DKIM) signing/verifying milter for Postfix/Sendmail."
 
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+
+class FileMacroExpand(distutils.cmd.Command):
+    description = "Expand @@ variables in input files, simlar to make macros."
+    user_options = [
+        ('sysconfigdir=', 'e', 'Specify system configuration directory. [/usr/local/etc]'),
+        ('sbindir=', 's', 'Specify system binary directory. [/usr/local/sbin]'),
+        ('bindir=', 'b', 'Specify binary directory. [/usr/loca/bin]'),
+        ('rundir=', 'r', 'Specify run state directory. [/run]'),
+    ]
+
+    def initialize_options(self):
+        self.sysconfigdir = '/usr/local/etc'
+        self.sbindir = '/usr/local/sbin'
+        self.bindir = '/usr/local/bin'
+        self.rundir = '/run'
+
+    def finalize_options(self):
+        self.configdir = self.sysconfigdir + '/dkimpy-milter'
+        self.rundir += '/dkimpy-milter'
+
+    def run(self):
+        files = ['etc/dkimpy-milter.conf', 'man/dkimpy-milter.conf.5', \
+                 'system/dkimpy-milter.service', 'system/dkimpy-milter', \
+                 'system/dkimpy-milter.openrc', \
+                 'system/socket-activation/dkimpy-milter.service', \
+                 'system/socket-activation/dkimpy-milter.socket', ]
+        for infile in files:
+            outfile = ''
+            try:
+                filein = open(infile + '.in')
+                for line in filein:
+                    for function in ["@SYSCONFDIR@", "@CONFDIR@", "@SBINDIR@", "@BINDIR@", "@RUNSTATEDIR@"]:
+                        splitline = line.split(function)
+                        if len(splitline) > 1:
+                            if function == "@SYSCONFDIR@":
+                                line = splitline[0] + self.sysconfigdir + splitline[1]
+                            elif function == "@CONFDIR@":
+                                line = splitline[0] + self.configdir + splitline[1]
+                            elif function == "@SBINDIR@":
+                                line = splitline[0] + self.sbindir + splitline[1]
+                            elif function == "@BINDIR@":
+                                line = splitline[0] + self.bindir + splitline[1]
+                            elif function == "@RUNSTATEDIR@":
+                                line = splitline[0] + self.rundir + splitline[1]
+                    outfile += line
+                out = open(infile, 'w')
+                for line in outfile:
+                    out.write(line)
+                out.close()
+            except FileNotFoundError as x:
+                pass
+
 kw = {}  # Work-around for lack of 'or' requires in setuptools.
 try:
-    import DNS
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'PyDNS']
+    import dns
+    kw['install_requires'] = ['dkimpy>=1.0', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'dnspython>=1.16.0']
 except ImportError:  # If PyDNS is not installed, prefer dnspython
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'dnspython']
+    kw['install_requires'] = ['dkimpy>=1.0', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'Py3DNS']
 
 setup(
     name='dkimpy-milter',
-    version='1.0.3',
+    version='1.2.1',
     author='Scott Kitterman',
     author_email='scott@kitterman.com',
     url='https://launchpad.net/dkimpy-milter',
     description=description,
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     download_url = "https://pypi.python.org/pypi/dkimpy-milter",
     classifiers= [
         'Development Status :: 5 - Production/Stable',
@@ -43,7 +104,7 @@ setup(
         'License :: OSI Approved :: GNU General Public License (GPL)',
         'Natural Language :: English',
         'Operating System :: POSIX',
-        'Programming Language :: Python :: 2 :: Only',
+        'Programming Language :: Python :: 3 :: Only',
         'Topic :: Communications :: Email :: Mail Transport Agents',
         'Topic :: Communications :: Email :: Filters',
         'Topic :: Security',
@@ -57,10 +118,14 @@ setup(
     include_package_data=True,
     data_files=[(os.path.join('share', 'man', 'man5'),
         ['man/dkimpy-milter.conf.5']), (os.path.join('share', 'man', 'man8'),
-        ['man/dkimpy-milter.8']), ('etc', ['etc/dkimpy-milter.conf']),
-        (os.path.join('lib', 'systemd', 'system'),
+        ['man/dkimpy-milter.8']), (os.path.join('etc', 'dkimpy-milter'),
+        ['etc/dkimpy-milter.conf']), (os.path.join('lib', 'systemd', 'system'),
         ['system/dkimpy-milter.service']),(os.path.join('etc', 'init.d'),
-        ['system/dkimpy-milter'])],
+        ['system/dkimpy-milter']), (os.path.join('etc', 'init.d'),
+        ['system/dkimpy-milter.openrc'])],
     zip_safe = False,
+    cmdclass={
+        'expand': FileMacroExpand,
+    },
     **kw
 )

system/dkimpy-milter

@@ -18,10 +18,8 @@
 # Short-Description: dkimpy-milter
 # Description:       Python DKIM Milter for Sendmail and Postfix
 ### END INIT INFO
-prefix="/usr/local"
-exec_prefix=${prefix}
-sysconfdir="/usr/local/etc"
-bindir="${exec_prefix}/bin/"
+sysconfdir="/usr/local/etc/dkimpy-milter"
+bindir="/usr/local/bin"
 RUNDIR="/run/dkimpy-milter"
 DAEMON=${bindir}/dkimpy-milter
 PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
@@ -35,8 +33,8 @@ test -x $DAEMON || exit 0
 
 # Include dkimpy-python defaults if available
 # Typically not used
-if [ -f /etc/default/dkimpy-milter ] ; then
-	. /etc/default/dkimpy-milter
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
+	. $sysconfdir/default/dkimpy-milter
 fi
 
 set -e
@@ -120,7 +118,7 @@ case "$1" in
         echo "$NAME."
 	;;
   status)
-        status_of_proc -p /var/run/dkimpy-milter/dkimpy-milter.pid /usr/local/bin/dkimpy-milter dkimpy-milter
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
         ;;
 
   *)

system/dkimpy-milter.in

@@ -0,0 +1,131 @@
+#! /bin/sh
+#
+# skeleton	example file to build /etc/init.d/ scripts.
+#		This file should be used to construct scripts for /etc/init.d.
+#
+#		Written by Miquel van Smoorenburg <miquels@cistron.nl>.
+#		Modified for Debian 
+#		by Ian Murdock <imurdock@gnu.ai.mit.edu>.
+#
+# Version:	@(#)skeleton  1.9  26-Feb-2001  miquels@cistron.nl
+#
+### BEGIN INIT INFO
+# Provides:          dkim-milter dkim-milter-python dkimpy-milter
+# Required-Start:    $remote_fs $syslog $network $time
+# Required-Stop:     $remote_fs $syslog $network
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: dkimpy-milter
+# Description:       Python DKIM Milter for Sendmail and Postfix
+### END INIT INFO
+sysconfdir="@CONFDIR@"
+bindir="@BINDIR@"
+RUNDIR="@RUNSTATEDIR@"
+DAEMON=${bindir}/dkimpy-milter
+PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
+NAME=dkimpy-milter
+DESC="Python DKIM Milter"
+USER=dkimpy-milter
+GROUP=dkimpy-milter
+SOCKET=$RUNDIR/dkimpy-milter.sock
+
+test -x $DAEMON || exit 0
+
+# Include dkimpy-python defaults if available
+# Typically not used
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
+	. $sysconfdir/default/dkimpy-milter
+fi
+
+set -e
+
+. /lib/lsb/init-functions
+
+case "$1" in
+  start)
+	echo -n "Starting $DESC: "
+	# Create the run directory if it doesn't exist
+	if [ ! -d $RUNDIR ]; then
+		install -o $USER -g $GROUP -m 755 -d $RUNDIR || return 2
+	fi
+
+	# Clean up stale sockets
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		pid=`cat $RUNDIR/$NAME.pid`
+		if ! ps -C $DAEMON -s $pid >/dev/null; then
+			rm $RUNDIR/$NAME.pid
+			# UNIX sockets may be specified with or without the
+			# local: prefix; handle both
+			t=`echo $SOCKET | cut -d: -f1`
+			s=`echo $SOCKET | cut -d: -f2`
+			if [ -e $s -a -S $s ]; then
+				if [ "$t" = "$s" -o "$t" = "local" ]; then
+					rm $s
+				fi
+			fi
+		fi
+	fi
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+	echo "$NAME."
+	;;
+  stop)
+	echo -n "Stopping $DESC: "
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		chown root:root $RUNDIR/$NAME.pid
+		start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid 
+		rm $RUNDIR/$NAME.pid
+		#echo $SOCKET
+		if [ -e $SOCKET ]; then
+			rm $SOCKET
+		fi
+	fi
+	echo "$NAME."
+	;;
+  force-reload)
+        echo -n "Force reloading $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        sleep 1
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+        ;;
+  restart)
+        echo "Restarting $DESC: "
+        echo -n "Stopping $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        echo "$NAME."
+	sleep 1
+        echo -n "Starting $DESC: "
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+	;;
+  status)
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
+        ;;
+
+  *)
+	N=/etc/init.d/$NAME
+	echo "Usage: $N {start|stop|force-reload|restart|}" >&2
+	exit 1
+	;;
+esac
+
+exit 0

system/dkimpy-milter.openrc

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="/usr/local/etc/dkimpy-milter/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="/usr/local/bin/dkimpy-milter"
+pidfile="/run/dkimpy-milter/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.openrc.in

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="@CONFDIR@/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="@BINDIR@/dkimpy-milter"
+pidfile="@RUNSTATEDIR@/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.service

@@ -5,8 +5,8 @@ After=network.target
 
 [Service]
 Type=simple
-PIDFile=/var/run/dkimpy-milter/dkimpy-milter.pid
-ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter.conf 
+PIDFile=/run/dkimpy-milter/dkimpy-milter.pid
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf -P /run/dkimpy-milter/dkimpy-milter.pid
 
 [Install]
 WantedBy=multi-user.target

system/dkimpy-milter.service.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5) 
+After=network.target
+
+[Service]
+Type=simple
+PIDFile=@RUNSTATEDIR@/dkimpy-milter.pid
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf -P @RUNSTATEDIR@/dkimpy-milter.pid
+
+[Install]
+WantedBy=multi-user.target

system/socket-activation/README.md

@@ -0,0 +1,32 @@
+This directory contains example systemd unit files for running a
+supervised, socket-activated instance of dkimpy-milter.
+
+There are several advantages of using socket activation:
+
+- dkimpy-milter never runs with elevated privileges, they are dropped
+  before any dkimpy-milter code is executed.
+   
+- The socket is opened before dkimpy-milter runs.  This means that
+  clients can connect() to the socket immediately.  So even if there
+  is a delay in dkimpy-milter startup, or in libmilter itself, the
+  connection will not fail.
+  
+- You can set the privileges of a listening Unix-domain socket by an
+  override of ListenGroup= in dkimpy-milter.socket (see
+  systemd.unit(5) for how to override).  This lets you control who has
+  access to the daemon with finer granularity than is available with
+  dkimpy-milter on its own.
+  
+- dkimpy-milter will not consume system resources if it is not used.
+
+- A fully-supervised dkimpy-milter needs no PIDFile, UMask, UserID, or
+  Socket configuation.  This eliminates common race conditions and
+  startup failures, and simplifies the resulting configuration file.
+  
+There is one downside to using socket activation:
+
+- it will only work on systems where libmilter can support connection
+  strings like "fd:3".  This has been supported on Debian and derived
+  systems since sendmail 8.14.4-6 (before Debian Jessie, in early
+  2014), see for example:
+  https://sources.debian.org/src/sendmail/8.15.2-8/debian/patches/socket_activation.patch/

system/socket-activation/dkimpy-milter.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.service.in

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.socket

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=/run/dkimpy-milter/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

system/socket-activation/dkimpy-milter.socket.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=@RUNSTATEDIR@/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

tests/00_minimal.miltertest

@@ -0,0 +1,12 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." complete")
+   end
+end

tests/01_connect.miltertest

@@ -0,0 +1,40 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+         error("mt.conninfo() failed "..keytype.." "..func)
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.conninfo() unexpected reply "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_ADDHDRS) then
+         print("could add headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not add headers "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_CHGHDRS) then
+         print("could change headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not change headers "..keytype.." "..func)
+      end
+
+-- -- FIXME: this part of the test fails, as apparently the
+-- -- dkimpy-milter claims the right to change the body of a message,
+-- -- even though it shouldn't.  How can we fix the negotiation?
+--      if mt.test_action(conn, SMFIF_CHGBODY) then
+--         error("mt.test_action() says could change body "..keytype.." "..func)
+--      else
+--         print("could not change body "..keytype.." "..func)
+--      end
+
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." test complete")
+   end
+end

tests/02_sign_message.miltertest

@@ -0,0 +1,100 @@
+-- -*- lua -*-
+
+msg = {
+   ['headers'] = {
+      ['From'] = 'Alice <alice@example.net>',
+      ['Message-Id'] = '<dkimpy-milter-test-02@example.net>',
+      ['To'] = 'Bob <bob@example.biz>',
+      ['Date'] = 'Mon, 18 Feb 2019 08:32:50 -0500',
+      ['Subject'] = 'Signing test',
+      ['Content-Type'] = 'text/plain',
+   },
+   ['body'] = "This is a test!\r\n",
+}
+
+-- returns miltertest connection object
+function connect_and_send (sockname, headers, body)
+   conn = mt.connect(sockname)
+   if conn == nil then
+      error "mt.connect() failed"
+   end
+   if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+      error "mt.conninfo() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.conninfo() unexpected reply"
+   end
+
+   -- mt.macro(conn, SMFIC_MAIL, "i", "simple-message")
+   if mt.mailfrom(conn, "<alice@example.net>") ~= nil then
+      error "mt.mailfrom() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.mailfrom() unexpected reply"
+   end
+   -- mt.rcptto() is called implicitly
+
+   -- send headers
+   for key,value in pairs(headers) do
+      if mt.header(conn, key, value) ~= nil then
+         error("mt.header(" .. key .. ") failed")
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.header(" .. key .. ") unexpected reply")
+      end
+   end
+   -- send EOH
+   if mt.eoh(conn) ~= nil then
+      error "mt.eoh() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.eoh() unexpected reply"
+   end
+
+   -- send body
+   if mt.bodystring(conn, body) ~= nil then
+      error "mt.bodystring() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.bodystring() unexpected reply"
+   end
+   -- end of message; let the filter react
+   if mt.eom(conn) ~= nil then
+      error "mt.eom() failed"
+   end
+   reply = mt.getreply(conn)
+   if reply ~= SMFIR_CONTINUE then
+      error ("mt.eom() unexpected reply: " .. reply)
+   end
+   return conn
+end
+
+for _, keytype in ipairs({"ed25519", "rsa", "ed25519.stable", "rsa.stable", "ed25519.table", "rsa.table"}) do
+   mt.echo("testing "..keytype)
+   signing = connect_and_send("unix:"..keytype..".signing.sock", msg.headers, msg.body)
+   -- verify that a test header field got added
+   if not mt.eom_check(signing, MT_HDRINSERT) then
+      error "no header added by signer"
+   end
+
+   signature = mt.getheader(signing, "DKIM-Signature", 0)
+
+   mt.disconnect(signing)
+
+   mt.echo("DKIM-Signature: " .. signature)
+
+   msg.headers['DKIM-Signature'] = signature
+
+   verify = connect_and_send("unix:"..keytype..".verify.sock", msg.headers, msg.body)
+
+   if not mt.eom_check(verify, MT_HDRINSERT) then
+      error "no header added in verify"
+   end
+
+   authres = mt.getheader(verify, "Authentication-Results", 0)
+   mt.echo("Authentication-Results: "..authres)
+
+   mt.disconnect(verify)
+
+   mt.echo(keytype.." complete")
+end

tests/dkimpy-milter

@@ -0,0 +1,2 @@
+#!/bin/sh
+python3 -m dkimpy_milter "$@"

tests/runtests

@@ -0,0 +1,143 @@
+#!/bin/bash
+
+set -e
+WORKDIR=$(mktemp -d)
+TESTDIR=$(realpath "$(dirname "$0")")
+DKIMPY_MILTER=${DKIMPY_MILTER:-"$TESTDIR/dkimpy-milter"}
+KEY_TYPES=(ed25519 rsa)
+
+cd "$WORKDIR"
+
+printf "Testing %s from directory %s\n" "$DKIMPY_MILTER" "$WORKDIR"
+
+for keytype in "${KEY_TYPES[@]}"; do
+    dknewkey --ktype "$keytype" "testkey.$keytype"
+    if [ "$keytype" = ed25519 ]; then
+        keyfile=KeyFileEd25519
+        selector=SelectorEd25519
+    elif [ "$keytype" = rsa ]; then
+        keyfile=KeyFile
+        selector=Selector
+    fi
+    if [ "$keytype" = ed25519 ]; then
+        keytable=KeyTableEd25519
+        signingtable=SigningTable
+        selector=SelectorEd25519
+    elif [ "$keytype" = rsa ]; then
+        keytable=KeyTable
+        signingtable=SigningTable
+        selector=Selector
+    fi
+    cat > "$keytype.signing.conf" <<EOF
+Domain example.net
+$keyfile testkey.$keytype.key
+$selector testkey
+Socket unix:$keytype.signing.sock
+PidFile $keytype.signing.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.verify.conf" <<EOF
+Socket unix:$keytype.verify.sock
+PidFile $keytype.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+MinimumKeyBits 2048
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.conf" <<EOF
+$keyfile testkey.$keytype.key
+$selector testkey
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.stable.signing.sock
+PidFile $keytype.stable.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.verify.conf" <<EOF
+Socket unix:$keytype.stable.verify.sock
+PidFile $keytype.stable.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.table.conf" <<EOF
+$keytable $WORKDIR/$keytype-table
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.table.signing.sock
+PidFile $keytype.table.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+cat > "$keytype.table.verify.conf" <<EOF
+Socket unix:$keytype.table.verify.sock
+PidFile $keytype.table.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat >  "$keytype-table" <<EOF
+preskey example.org:testkey:$WORKDIR/testkey.$keytype.key
+orgkey  example.org:testkey:$WORKDIR/testkey.$keytype.key
+netkey  example.net:testkey:$WORKDIR/testkey.$keytype.key
+EOF
+
+    cat > "signing-table" <<EOF
+president@example.org	@special.example.org:preskey
+*@example.org		orgkey
+*@example.net		netkey
+EOF
+done
+
+cleanup() {
+    echo cleaning up jobs:
+    jobs
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify stable stable.verify table table.verify; do
+            if [ -s "$keytype.$func.pid" ] && kill -0 "$(cat "$keytype.$func.pid")"; then
+                kill "$(cat $keytype.$func.pid)"
+            fi
+        done
+    done
+    wait
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify stable stable.verify table table.verify; do
+            errdata="$keytype.$func.stderr"
+            if [ -s "$errdata" ]; then
+                printf -- "-> %s:\n" "$errdata"
+                cat "$errdata"
+                printf -- "-> end %s\n" "$errdata"
+            fi
+        done
+    done
+    rm -rf "$WORKDIR"
+}
+
+for keytype in "${KEY_TYPES[@]}"; do
+    for func in signing verify stable stable.verify table table.verify; do
+        PYTHONPATH="$(dirname "$TESTDIR")" "$DKIMPY_MILTER" "$keytype.$func.conf" 2>"$keytype.$func.stderr" &
+    done
+done
+trap cleanup EXIT
+
+# ugly ugly (how are we supposed to know that the milters are all ready?):
+sleep 2
+
+# uses miltertest from opendkim:
+for x in ${TESTS:-"$TESTDIR"/*.miltertest}; do
+    if ! [ -e "$x" ]; then
+        if [ -e "$TESTDIR/$x" ]; then
+            x="$TESTDIR/$x"
+        fi
+    fi
+    printf -- "-> running %s...\n" "$x"
+    miltertest -s "$x"
+done
+
+rm -rf "$(dirname $TESTDIR)/dkimpy_milter/__pycache__"