Update version/release date for 1.1.2

(LP: #1844189)

.gitignore

@@ -1,3 +1,4 @@
 dist
 dkimpy_milter.egg-info
 *.pyc
+*~

CHANGES

@@ -1,3 +1,59 @@
+1.1.2 2019-09-23
+ - Fix variable initialization so mailformed mails missing body From do not
+   cause a traceback (LP: #1844161)
+ - Catch more ascii encoding errors to improve resilience against bad data
+   (LP: #1844189)
+
+1.1.1 2019-09-06
+ - Fix startup logging so it provides information at a useful time
+ - Fix verify processing so missing (optional) i= tag doesn't cause the milter
+   to fail (LP: #1842250)
+ - Fix message extraction so that signing in the same pass through the milter
+   as verifying works correctly
+
+1.1.0 2019-04-12
+ - Add SubDomains option to enable signing for sub-domains (LP: #1811535)
+ - Port to python3 (LP: #1815502)
+ - Add test suite using opendkim miltertest
+ - When Socket is absolute path, do not strip leading /
+ - Handle unix: socket prefix the same as local:
+ - Set up correct AuthservID defaults
+ - config: Reassemble strings sensibly
+ - Consistently prefer dnspython to Py3DNS (LP: #1815558)
+
+1.0.1 2019-02-11
+ - Reorder milter start and dropping privileges so permissions on Unix socket
+   are correct (LP: 1797720)
+ - Make domain checks case insensitive for determining if signing should be
+   done (LP: #1815311)
+ - Add additional Sendmail configuration information to README from OpenDKIM
+   update based on input from Дилян Палаузов (LP: #1801619)
+ - Add information on Ed25519 key creation to README (LP: #1815313)
+
+1.0.0 2018-05-11
+ - Minor documentation updates
+ - Deleted reference to obsolete syslog target in unit file
+
+0.9.7 2018-03-19
+ - Made sysv init executable
+ - Add missing documentation key to system/dkimpy-milter.service
+ - Put version directly in setup.py and do not import dkimpy_milter to ease
+   install via pip
+ - Minor sysv init improvments
+
+0.9.6 2018-03-13
+ - Fixed typo in package installation section of README
+ - Added more to README about first run with systemd
+ - Fixed typo in path for fallback location of the config file if one is not
+   provided
+ - Added protection for malformed From addresses.  If the From does not at
+   least have an '@' in the address, then the signing domain is not extracted
+   and the message will not be signed
+
+0.9.5.1 2018-03-10
+ - Add conf file location to systemd unit file
+ - Fix setup.py install locations so they are installed correctly
+
 0.9.5 2018-03-10
  - Beta 1 (updated Alpha -> Beta warning in README and trove classifiers)
  - Added support for MacroList option

README

@@ -1,23 +1,29 @@
-This is a DKIM signing and verification milter.  In theory it works with both
+OVERVIEW
-Postfix and Sendmail, but the author has zero experience with Sendmail, so
+========
-reports of success/failure with Sendmail and patches are welcom.
+
+This is a DKIM signing and verification milter.  It has been tested with both
+Postfix and Sendmail.
 
 The configuration file is designed to be compatible with OpenDKIM, but only
 a subset of OpenDKIM options are supported.  If an unsupported option is
 specified, an error will be raised.
 
+
+INSTALLATION
+===========
+
 This package includes a default configuration file and man pages.  For those
 to be installed when installing using setup.py, the following incantation is
 required because setuptools developers decided not being able to do this by
 default is a feature:
 
-python setup.py install --single-version-externally-managed --record=/dev/null
+python3 setup.py install --single-version-externally-managed --record=/dev/null
 
-For users of Debian Stable (Debian 9, Codename Squueze), all dependencies are
+For users of Debian Stable (Debian 9, Codename Squeeze), all dependencies are
 available in either the main or backports repositories:
 
-[sudo] apt install python-milter python-nacl pthon-ipaddress python-dnspython
+[sudo] apt install python3-milter python3-nacl python3-dnspython
-[sudo] apt install -t squeeze-backports python-authres python-dkim
+[sudo] apt install -t stretch-backports python3-authres python3-dkim
 
 The preferred method of installation is from PyPi using pip (if distribution
 packages are not available):
@@ -25,12 +31,57 @@ packages are not available):
 [sudo] pip install dkimpy_milter
 
 Using pip will cause required packages to be installed via easy_install if they
-have not been previously installed.
+have not been previously installed.  Because pymilter and PyNaCl are compiled
+Python extensions, the system will need appropriate development packages and
+an C compiler.  Alternately, install these dependencies from distribution/OS
+packages and then pip install dkimpy_milter.
 
-The milter will work with either pydns (DNS) or dnspython (dns), preferring
+The milter will work with either py3dns (DNS) or dnspython (dns), preferring
 dnspython is both are available.  The dkimpy DKIM module also works with
 either.
 
+
+SETUP
+====
+
+SIGNING KEYS
+============
+
+In order to create DKIM signatures, a private key must be available.  Signing
+keys should be protected (owned by root:root with permissions 600 in a
+directory that is not world readable).  Different keys are required for RSA
+and (if used) Ed25519.
+
+RSA
+===
+
+Both public and private keys for RSA have standard formats and there are many
+tools available to create them.  Keys must (RFC 8302) have a minimum size of
+1024 bits and should have a size of at least 2048 bits.  The dknewkey script
+that is provided with dkimpy is one such tool:
+
+dknewkey exampleprivkey
+
+will produce both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  RSA is the default key
+type.  2048 bits is the default key size.
+
+ED25519
+=======
+
+There is no standardized non-binary representation for Ed25519 private keys,
+so in order to generate Ed25519 keys for dkimpy-milter, dkimpy specific tools
+must be used to be compatible.  The same dknewkey script support Ed25519:
+
+dknewkey --ktype ed25519 anothernewkey
+
+will provide both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  Ed25519 keys do not have
+variable bit lengths.
+
+MTA INTEGRATION
+==============
+
 Both a systemd unit file and a sysv init file are provided.  Both make
 assumptions about defaults being used, e.g. if a non-standard pidfile name is
 used, they will need to be updated.  The sysv init file is Debian specific and
@@ -43,21 +94,30 @@ As an example, using the default dkimpy-user on Debian, the command would be:
 
 [sudo] adduser --system --no-create-home --quiet --disabled-password \
                --disabled-login --shell /bin/false --group \
-               --home /var/run/dkimpy-milter dkimpy-milter
+               --home /run/dkimpy-milter dkimpy-milter
 
 Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
 missing, the milter will create it on startup.
 
+To start dkimpy-milter with systemd for the first time, you will need to take
+the following steps:
+
+[sudo] systemctl daemon-reload
+[sudo] systemctl enable dkimpy-milter
+[sudo] systemctl start dkimpy-milter
+[sudo] systemctl status dkimpy-milter (to verify it started correctly)
+
 As with all milters, dkimpy-milter needs to be integrated with your MTA of
 choice (Sendmail or Postfix).
 
-For Sendmail:
+SENDMAIL
+========
 
 Configuration is very similar to opendkim, but needs some adjustment for
 dkimpy-milter. Here's an example configuration line to include in your
 sendmail.mc:
 
-INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/var/run/dkimpy-milter/dkimpy-milter.sock')dnl
+INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/run/dkimpy-milter/dkimpy-milter.sock')dnl
 
 Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
 and a restart of sendmail.  Note that S= needs to match the value of Socket in
@@ -67,12 +127,64 @@ Milter support should be present by default in most versions of sendmail
 these days, but if not included in your Sendmail build, see:
 http://www.elandsys.com/resources/sendmail/milter.html
 
-For Postfix:
+ISSUES USING SENDMAIL TO SIGN AND VERIFY
+========================================
+
+When using the sendmail MTA in both signing and verifying mode, there are
+a few issues of which to be aware that might cause operational problems
+and deserve consideration.
+
+(a) When the MTA will be used for relaying emails, e.g. delivering to other
+    hosts using the aliases mechanism, it is important not to break
+    signatures inserted by the original sender.  This is particularly sensitive
+    particular when the sending domain has published a "reject" DMARC policy.
+
+    By default, sendmail quotes to address header fields when there are no
+    quotes and the display part of the address contains a period or an
+    apostrophe.  However, opendkim only sees the raw, unmodified form of
+    the header field, and so the content that gets verified and what gets
+    signed will not be the same, guaranteeing the attached signature is not
+    valid.
+
+    To direct sendmail not to modify the headers, add this to your sendmail.mc:
+
+    	conf(`confMUST_QUOTE_CHARS', `')
+
+(b) As stated in sendmail's KNOWNBUGS file, sendmail truncates header field
+    values longer than 256 characters, which could mean truncating the domain
+    of a long From: header field value and invalidating the signature.
+    You may wish to consider increasing MAXNAME in sendmail/conf.h to mitigate
+    changing the messages and invalidating their signatures.  This change
+    requires recompiling sendmail.
+
+(c) Similar to (a) above, sendmail may wrap very long single-line recipient
+    fields for presentation purposes; for example:
+
+    To: very long name <a@example.org>,anotherloo...ong name b <b@example.org>
+
+    ...might be rewritten as:
+
+    To: very long name <a@example.org>,
+    	anotherloo...ong name b <b@example.org>
+
+    This rewrite is also done after opendkim has seen the message, meaning
+    the signature opendkim attaches to the message does not match the
+    content it signed.  There is not a known configuration change to
+    mitigate this mutation.
+
+    The only known mechanism for dealing with this is to have distinct
+    instances of opendkim do the verifying (inbound) and signing (outbound)
+    so that the version that arrives at the signing instance is already
+    in the rewritten form, guaranteeing the input and output are the same
+    and thus the signature matches the payload.
+
+POSTFIX
+=======
 
 Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
-README_FILES/MILTER_README).  Here's an example master.cf excerpt the talks to
+README_FILES/MILTER_README).  Here's an example master.cf excerpt that talks
-two dkimpy-milter instances, one configured for signing and one configured for
+to two dkimpy-milter instances, one configured for signing and one configured
-verification:
+for verification:
 
 smtp       inet  n       -       -       -      -       smtpd
     ...
@@ -117,11 +229,15 @@ MacroListVerify		daemon_name|VERIFYING
 ...
 
 
-The python DKIM library, dkimpy, requires the entire message being signed or
+NOTES
-verified to be in memory, so dkimpy-milter does not write messages out to a temp
+=====
-file.  This may impact performance on low-memory systems.
 
-This is an beta grade release to support interoperability testing with Ed25519
+The python DKIM library, dkimpy, requires the entire message being signed or
-signatures sufficient functionality for basic use.  The documented
+verified to be in memory, so dkimpy-milter does not write messages out to a
-functionality has been implemented and at least partially tested.  It is free
+temp file.  This may impact performance on low-memory systems.
-of known major defects, but is not fully tested in a variety of environments.
+
+DKIM with Ed25519 signatures are described in RFC 8463.  Version 1.0.0 and
+later support Ed25519 signing and verification.  RFC 8301 removed rsa-sha1
+from DKIM.  dkimpy-milter does not sign with rsa-sha1, but still considers
+rsa-sha1 signatures as valid for verification because they are still in
+common use and are not known to be cryptographically broken.

TODO

@@ -39,20 +39,32 @@ MacroListVerify	implemented	verified
 SyslogSuccess   implemented     verified
 
 1.0.0
-No additional features planned
+No additional features
 
-Plannedataset type support (if needed):
+1.0.1
-db:/.db
+Bug fix only, improved documentation
+
+1.1.0
+Port to Python 3  implemented	verified
+Subdomain support implemented	verified
+Test suite	  implemented	verified
+
+Planned dataset type support (if needed):
 mdb:
 
 Considered for near-term feature release
 
+KeyTable
+KeytableEd25519
+SigningTable
+SigningTableEd25519
 AlwaysAddARHeader
 ChangeRootDirectory
 ClockDrift (requires dkimpy change)
-DNSTimeout (requires dkmpy change)
+DNSTimeout (requires dkimpy change)
 MilterDebug
 MinimumKeyBits
+OversignHeaders (may require dkimpy changes)
 PeerList
 SignatureAlgorithm
 
@@ -66,8 +78,6 @@ ExternalIgnoreList
 FixCRLF
 KeepAuthResults
 KeepTemporaryFiles
-KeyTable
-KeytableEd25519
 LogResults
 LogWhy
 MaximumHeaders
@@ -83,7 +93,6 @@ On-InternalError
 On-KeyNotFound
 On-NoSignature
 On-SignatureError
-OversignHeaders
 RemoveARAll
 RemoveARFrom
 RemoveOldSignatures 
@@ -92,7 +101,6 @@ RequireSafeKeys
 SignatureAlgorithm
 SignatureTTL
 SignHeaders
-SigningTable
 SoftwareHeader
 StrictHeaders
 SubDomains

dkimpy_milter/__init__.py

@@ -1,4 +1,4 @@
-#! /usr/bin/python2
+#! /usr/bin/python3
 # Original dkim-milter.py code:
 # Author: Stuart D. Gathman <stuart@bmsi.com>
 # Copyright 2007 Business Management Systems, Inc.
@@ -28,8 +28,9 @@ import dkim
 import authres
 import os
 import tempfile
-import StringIO
+import io
 import re
+import codecs
 from Milter.utils import parse_addr, parseaddr
 import dkimpy_milter.config as config
 from dkimpy_milter.util import drop_privileges
@@ -39,7 +40,7 @@ from dkimpy_milter.util import read_keyfile
 from dkimpy_milter.util import own_socketfile
 from dkimpy_milter.util import fold
 
-__version__ = "0.9.5"
+__version__ = "1.0.1"
 FWS = re.compile(r'\r?\n[ \t]+')
 
 
@@ -54,6 +55,7 @@ class dkimMilter(Milter.Base):
         self.privatersa = privateRSA
         self.privateed25519 = privateEd25519
         self.fp = None
+        self.fdomain = ''
 
     @Milter.noreply
     def connect(self, hostname, unused, hostaddr):
@@ -61,7 +63,9 @@ class dkimMilter(Milter.Base):
         self.external_connection = False
         self.hello_name = None
         # sometimes people put extra space in sendmail config, so we strip
-        self.receiver = self.getsymval('j').strip()
+        self.receiver = self.getsymval('j')
+        if self.receiver is not None:
+            self.receiver = self.receiver.strip()
         try:
             self.AuthservID = milterconfig['AuthservID']
         except:
@@ -108,7 +112,7 @@ class dkimMilter(Milter.Base):
     def envfrom(self, f, *str):
         if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 2:
             syslog.syslog("mail from: {0} {1}".format(f, str))
-        self.fp = StringIO.StringIO()
+        self.fp = io.BytesIO()
         self.mailfrom = f
         t = parse_addr(f)
         if len(t) == 2:
@@ -130,20 +134,27 @@ class dkimMilter(Milter.Base):
             self.has_dkim += 1
         if lname == 'from':
             fname, self.author = parseaddr(val)
-            self.fdomain = self.author.split('@')[1]
+            try:
+                self.fdomain = self.author.split('@')[1].lower()
+            except IndexError as er:
+                pass # self.author was not a proper email address
             if (milterconfig.get('Syslog') and
                     milterconfig.get('debugLevel') >= 1):
                 syslog.syslog("{0}: {1}".format(name, val))
         elif lname == 'authentication-results':
             self.arheaders.append(val)
         if self.fp:
-            self.fp.write("%s: %s\n" % (name, val))
+            try:
+                self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'ascii')))
+            except:
+                # Don't choke on header fields with non-ascii garbage in them.
+                pass
         return Milter.CONTINUE
 
     @Milter.noreply
     def eoh(self):
         if self.fp:
-            self.fp.write("\n")   # terminate headers
+            self.fp.write(b"\n")   # terminate headers
         self.bodysize = 0
         return Milter.CONTINUE
 
@@ -171,39 +182,40 @@ class dkimMilter(Milter.Base):
             except:
                 # Don't error out on unparseable AR header fiels
                 pass
-        # Check or sign DKIM
+        # Check and/or sign DKIM
         self.fp.seek(0)
+        txt = self.fp.read()
         if milterconfig.get('Domain'):
             domain = milterconfig.get('Domain')
         else:
             domain = ''
+        if milterconfig.get('SubDomains'):
+            self.fdomain = _get_parent_domain(self.fdomain, domain)
         if ((self.fdomain in domain) and not milterconfig.get('Mode') == 'v'
                 and not self.external_connection):
-            txt = self.fp.read()
             self.sign_dkim(txt)
         if ((self.has_dkim) and (not self.internal_connection) and
             (milterconfig.get('Mode') == 'v' or
              milterconfig.get('Mode') == 'sv')):
-            txt = self.fp.read()
             self.check_dkim(txt)
         if self.arresults:
             h = authres.AuthenticationResultsHeader(authserv_id=
                                                     self.AuthservID,
                                                     results=self.arresults)
-            h = fold(str(h))
+            h = fold(codecs.encode(str(h), 'ascii'))
             if (milterconfig.get('Syslog') and
                     milterconfig.get('debugLevel') >= 2):
-                syslog.syslog(str(h))
+                syslog.syslog(codecs.decode(h, 'ascii'))
-            name, val = str(h).split(': ', 1)
+            name, val = codecs.decode(h, 'ascii').split(': ', 1)
             self.addheader(name, val, 0)
         return Milter.CONTINUE
 
     def sign_dkim(self, txt):
-        canon = milterconfig.get('Canonicalization')
+        canon = codecs.encode(milterconfig.get('Canonicalization'), 'ascii')
         canonicalize = []
-        if len(canon.split('/')) == 2:
+        if len(canon.split(b'/')) == 2:
-            canonicalize.append(canon.split('/')[0])
+            canonicalize.append(canon.split(b'/')[0])
-            canonicalize.append(canon.split('/')[1])
+            canonicalize.append(canon.split(b'/')[1])
         else:
             canonicalize.append(canon)
             canonicalize.append(canon)
@@ -213,35 +225,36 @@ class dkimMilter(Milter.Base):
         try:
             if privateRSA:
                 d = dkim.DKIM(txt)
-                h = d.sign(milterconfig.get('Selector'), self.fdomain,
+                h = d.sign(codecs.encode(milterconfig.get('Selector'), 'ascii'), codecs.encode(self.fdomain, 'ascii'),
-                           privateRSA, canonicalize=(canonicalize[0],
+                           codecs.encode(privateRSA, 'ascii'),
+                           canonicalize=(canonicalize[0],
                                          canonicalize[1]))
-                name, val = h.split(': ', 1)
+                name, val = h.split(b': ', 1)
-                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
                 if (milterconfig.get('Syslog') and
                     (milterconfig.get('SyslogSuccess')
                      or milterconfig.get('debugLevel') >= 1)):
-                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b'a').decode(),
-                                                  d.signature_fields.get(b's'),
+                                                  d.signature_fields.get(b's').decode(),
-                                                  d.domain))
+                                                  d.domain.decode().lower()))
             if privateEd25519:
                 d = dkim.DKIM(txt)
-                h = d.sign(milterconfig.get('SelectorEd25519'), self.fdomain,
+                h = d.sign(codecs.encode(milterconfig.get('SelectorEd25519'), 'ascii'), codecs.encode(self.fdomain, 'ascii'),
                            privateEd25519, canonicalize=(canonicalize[0],
                                                          canonicalize[1]),
-                           signature_algorithm='ed25519-sha256')
+                           signature_algorithm=b'ed25519-sha256')
-                name, val = h.split(': ', 1)
+                name, val = h.split(b': ', 1)
-                self.addheader(name, val.strip().replace('\r\n', '\n'), 0)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
                 if (milterconfig.get('Syslog') and
                     (milterconfig.get('SyslogSuccess')
                      or milterconfig.get('debugLevel') >= 1)):
-                    syslog.syslog('{0}: {1} DKIM-Signature field added (s={2} '
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b'a').decode(),
-                                                  d.signature_fields.get(b's'),
+                                                  d.signature_fields.get(b's').decode(),
-                                                  d.domain))
+                                                  d.domain.decode().lower()))
         except dkim.DKIMException as x:
             if milterconfig.get('Syslog'):
                 syslog.syslog('DKIM: {0}'.format(x))
@@ -252,24 +265,27 @@ class dkimMilter(Milter.Base):
 
     def check_dkim(self, txt):
         res = False
+        self.header_a = None
         for y in range(self.has_dkim):  # Verify _ALL_ the signatures
             d = dkim.DKIM(txt)
             try:
+                dnsoverride = milterconfig.get('DNSOverride')
+                if isinstance(dnsoverride, str):
+                    syslog.syslog("DNSOverride: {0}".format(dnsoverride))
+                    res = d.verify(idx=y, dnsfunc=lambda _x: dnsoverride)
+                else:
                     res = d.verify(idx=y)
+                algo = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
                 if res:
-                    if d.signature_fields.get(b'a') == 'ed25519-sha256':
+                    if algo == 'ed25519-sha256':
                         self.dkim_comment = ('Good {0} signature'
-                                             .format(d.signature_fields
+                                             .format(algo))
-                                                     .get(b'a')))
                     else:
                         self.dkim_comment = ('Good {0} bit {1} signature'
-                                             .format(d.keysize,
+                                             .format(d.keysize, algo))
-                                                     d.signature_fields
-                                                     .get(b'a')))
                 else:
                     self.dkim_comment = ('Bad {0} bit {1} signature.'
-                                         .format(d.keysize,
+                                         .format(d.keysize, algo))
-                                                 d.signature_fields.get(b'a')))
             except dkim.DKIMException as x:
                 self.dkim_comment = str(x)
                 if milterconfig.get('Syslog'):
@@ -278,19 +294,31 @@ class dkimMilter(Milter.Base):
                 self.dkim_comment = str(x)
                 if milterconfig.get('Syslog'):
                     syslog.syslog("check_dkim: {0}".format(x))
-            self.header_i = d.signature_fields.get(b'i')
+            try:
-            self.header_d = d.signature_fields.get(b'd')
+                # i= is optional and dkimpy is fine if it's not provided
-            self.header_a = d.signature_fields.get(b'a')
+                self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
+            except TypeError as x:
+                self.header_i = None
+            try:
+                self.header_d = codecs.decode(d.signature_fields.get(b'd'), 'ascii')
+                self.header_a = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
+            except Exception as x:
+                self.dkim_comment = str(x)
+                if milterconfig.get('Syslog'):
+                    syslog.syslog("check_dkim: {0}".format(x))
+                self.header_d = None
+            if not self.header_a:
+                self.header_a = 'rsa-sha256'
             if res:
                 if (milterconfig.get('Syslog') and
                         (milterconfig.get('SyslogSuccess') or
                          milterconfig.get('debugLevel') >= 1)):
                     syslog.syslog('{0}: {1} DKIM signature verified (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
-                                                  d.signature_fields.get(b'a'),
+                                                  d.signature_fields.get(b'a').decode(),
-                                                  d.signature_fields.get(b's'),
+                                                  d.signature_fields.get(b's').decode(),
-                                                  d.domain))
+                                                  d.domain.decode().lower()))
-                self.dkim_domain = d.domain
+                self.dkim_domain = d.domain.lower()
             else:
                 if milterconfig.get('DiagnosticDirectory'):
                     fd, fname = tempfile.mkstemp(".dkim")
@@ -300,12 +328,18 @@ class dkimMilter(Milter.Base):
                         syslog.syslog('DKIM: Fail (saved as {0})'
                                       .format(fname))
                 else:
-                    syslog.syslog('DKIM: Fail ({0})'.format(d.domain))
+                    if milterconfig.get('Syslog'):
+                        if d.domain:
+                            syslog.syslog('DKIM: Fail ({0})'
+                                          .format(d.domain.lower()))
+                        else:
+                            syslog.syslog('DKIM: Fail, unextractable domain')
             if res:
                 result = 'pass'
             else:
                 result = 'fail'
             res = False
+            if self.header_d:
                 self.arresults.append(
                     authres.DKIMAuthenticationResult(result=result,
                                                  header_i=self.header_i,
@@ -314,8 +348,19 @@ class dkimMilter(Milter.Base):
                                                  result_comment=
                                                  self.dkim_comment)
             )
+            self.header_a = None
         return
 
+# get parent domain to be signed for if fdomain is a subdomain
+def _get_parent_domain(fdomain, domains):
+    for domain in domains:
+        rhs = '.'+domain
+        # compare right hand side of fdomain against .domain
+        if fdomain[-len(rhs):] == rhs:
+            # return parent domain on match
+            return domain
+    # or return the fdomain itself
+    return fdomain
 
 def main():
     # Ugh, but there's no easy way around this.
@@ -324,7 +369,7 @@ def main():
     global privateEd25519
     privateRSA = False
     privateEd25519 = False
-    configFile = '/etc/dkimpy-milter.conf'
+    configFile = '/usr/local/etc/dkimpy-milter.conf'
     if len(sys.argv) > 1:
         if sys.argv[1] in ('-?', '--help', '-h'):
             print('usage: dkimpy-milter [<configfilename>]')
@@ -345,13 +390,24 @@ def main():
     Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
     miltername = 'dkimpy-filter'
     socketname = milterconfig.get('Socket')
-    if milterconfig.get('Syslog'):
+    if socketname is None:
-        syslog.syslog('dkimpy-milter started:{0} user:{1}'
+        if int(os.environ.get('LISTEN_PID', '0')) == os.getpid():
-                      .format(pid, milterconfig.get('UserID')))
+            lfds = os.environ.get('LISTEN_FDS')
-    sys.stdout.flush()
+            if lfds is not None:
-    Milter.runmilter(miltername, socketname, 240)
+                if lfds != '1':
-    own_socketfile(milterconfig)
+                    syslog.syslog('LISTEN_FDS is set to "{0}", but we only know how to deal with "1", ignoring it'.
+                                  format(lfds))
+                else:
+                    socketname = 'fd:3'
+        if socketname is None:
+            socketname = 'local:/var/run/dkimpy-milter/dkimpy-milter.sock'
+    own_socketfile(milterconfig, socketname)
     drop_privileges(milterconfig)
+    sys.stdout.flush()
+    if milterconfig.get('Syslog'):
+        syslog.syslog('dkimpy-milter starting:{0} user:{1}'
+                      .format(pid, milterconfig.get('UserID')))
+    Milter.runmilter(miltername, socketname, 240)
 
 if __name__ == "__main__":
     main()

dkimpy_milter/__main__.py

@@ -0,0 +1,6 @@
+#!/usr/bin/python3
+
+from dkimpy_milter import main
+
+if __name__ == "__main__":
+    main()

dkimpy_milter/config.py

@@ -31,16 +31,16 @@ import stat
 import dkim
 import socket
 import ipaddress
-from dnsplug import Session
+from .dnsplug import Session
 
 #  default values
 defaultConfigData = {
     'Syslog': 'yes',
     'SyslogFacility': 'mail',
-    'UMask': 007,
+    'UMask': 0o07,
     'Mode': 'sv',
-    'Socket': 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
+    'Socket': None,
-    'PidFile': '/var/run/dkimpy-milter/dkimpy-milter.pid',
+    'PidFile': None,
     'UserID': 'dkimpy-milter',
     'Canonicalization': 'relaxed/simple',
     'InternalHosts': '127.0.0.1',
@@ -48,6 +48,8 @@ defaultConfigData = {
     'DiagnosticDirectory': '',
     'MacroList': '',
     'MacroListVerify': '',
+    'DNSOverride': None,
+    'SubDomains': False,
     'debugLevel': 0  # Undocumented config item for developer use
     }
 
@@ -84,14 +86,14 @@ class HostsDataset(object):
                 self.item = item[1:]
                 self.negative = True
             try:
-                self.item = ipaddress.ip_address(unicode(self.item, "utf-8"))
+                self.item = ipaddress.ip_address(str(self.item, "utf-8"))
                 if isinstance(self.item, ipaddress.IPv4Address):
                     self.isipv4 = True
                 elif isinstance(self.item, ipaddress.IPv6Address):
                     self.isipv6 = True
             except ValueError as e:
                 try:
-                    self.item = ipaddress.ip_network(unicode
+                    self.item = ipaddress.ip_network(str
                                                      (self.item, "utf-8"),
                                                      strict=False)
                     if isinstance(self.item, ipaddress.IPv4Network):
@@ -109,7 +111,7 @@ class HostsDataset(object):
 
     def match(self, connectip):
         '''Check if the connect IP is part of the dataset'''
-        source = ipaddress.ip_address(unicode(connectip, "utf-8"))
+        source = ipaddress.ip_address(str(connectip, "utf-8"))
         for item in self.dataset:
             if item.isdomain or item.ishostname:
                 result = self.matchname(source)   # Match host/domains first
@@ -159,13 +161,13 @@ class HostsDataset(object):
             if isinstance(source, ipaddress.IPv4Address):
                 ips = s.dns(name, 'A')
                 for ip in ips:
-                    ip = ipaddress.IPv4Address(unicode(ip, 'UTF-8'))
+                    ip = ipaddress.IPv4Address(str(ip, 'UTF-8'))
                     if ip == source:
                         results.append(name)
             if isinstance(source, ipaddress.IPv6Address):
                 ips = s.dns(name, 'AAAA')
                 for ip in ips:
-                    ip = ipaddress.IPv6Address(unicode(ip, 'UTF-8'))
+                    ip = ipaddress.IPv6Address(str(ip, 'UTF-8'))
                 if ip == source:
                     results.append(name)
         return results
@@ -224,13 +226,13 @@ def _processConfigFile(filename=None, configdata=None, useSyslog=1,
     '''Load the specified config file, exit and log errors if it fails,
     otherwise return a config dictionary.'''
 
-    import config
+    from . import config
     if configdata is None:
         configdata = config.defaultConfigData
     if filename is not None:
         try:
             _readConfigFile(filename, configdata)
-        except Exception, e:
+        except Exception as e:
             raise
             if useSyslog:
                 syslog.syslog(e.args[0])
@@ -294,19 +296,7 @@ def _dataset_to_list(dataset):
         else:
             return [dataset.strip().strip(',')]
         if dataset[-3:] == '.db' or dataset[:3] == 'db:':
-            #  This is a Sleepycat (Oracle) DB  dataset
+            #  This is a Sleepycat (Oracle) DB  dataset, which we dont support
-            import whichdb  # Will need rewriting someday for python3
-            if dataset[-3:] == '.db':
-                dbname = dataset
-            elif dataset[:3] == 'db:':
-                dbname = dataset[3:]
-            else:
-                raise dkim.ParameterError('Unimplmented dataset type: {0}'
-                                          .format(type(dataset)))
-            if whichdb.whichdb(dbname) != 'dbhash':
-                raise dkim.ParameterError('Unimplmented dataset type: {0}'
-                                          .format(type(dataset)))
-            #TODO replace this with code to use db maps
             raise dkim.ParameterError('Unsupported dataset db datase: {0}'
                                       .format(type(dataset)))
 
@@ -320,7 +310,9 @@ def _readConfigFile(path, configData=None, configGlobal={}):
     dictionary of name/value pairs based on configData and the values
     read from path.'''
 
-    debugLevel = configGlobal.get('debugLevel', 0)
+    # No config file data is available yet, so to debug _readConfigFile, set
+    # the value here.
+    debugLevel = 0
     if debugLevel >= 5:
         syslog.syslog('readConfigFile: Loading "%s"' % path)
     if configData is None:
@@ -336,6 +328,7 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         'PidFile': 'str',
         'UserID': 'str',
         'Domain': 'dataset',
+        'SubDomains': 'bool',
         'KeyFile': 'str',
         'KeyFileEd25519': 'str',
         'Selector': 'str',
@@ -346,13 +339,14 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         'DiagnosticDirectory': 'str',
         'MacroList': 'dataset',
         'MacroListVerify': 'dataset',
+        'DNSOverride': 'str',
         'debugLevel': 'int'
         }
 
     #  check to see if it's a file
     try:
         mode = os.stat(path)[0]
-    except OSError, e:
+    except OSError as e:
         syslog.syslog(syslog.LOG_ERR, 'ERROR stating "%s": %s'
                       % (path, e.strerror))
         return(configData)
@@ -388,9 +382,15 @@ def _readConfigFile(path, configData=None, configGlobal={}):
             value = data[1:]
 
         #  check validity of name
+        try:
             conversion = nameConversion.get(name)
+        except TypeError:
+            name = name[0]
+            syslog.syslog('Config item "%s" does not provide a value in file "%s"'
+                          % (name, path))
+            conversion = None
         if conversion is None:
-            syslog.syslog('ERROR: Unknown name "%s" in file "%s"'
+            syslog.syslog('ERROR: Unknown name or name missing value "%s" in file "%s"'
                           % (name, path))
             continue
 
@@ -400,6 +400,9 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         if conversion == 'bool':
             configData[name] = _find_boolean(value)
         elif conversion == 'str':
+            if isinstance(value, list):
+                configData[name] = line.split(None, 1)[1]
+            else:
                 configData[name] = str(value)
         elif conversion == 'int':
             configData[name] = int(value)
@@ -411,7 +414,7 @@ def _readConfigFile(path, configData=None, configGlobal={}):
             configData[name] = conversion(value)
     fp.close()
     try:
-        configData['AuthservID'] = _make_authserv_id(configData['AuthservID'])
+        configData['AuthservID'] = _make_authserv_id(configData.get('AuthservID', 'HOSTNAME'))
         configData['IntHosts'] = HostsDataset(configData['InternalHosts'])
     except:
         pass

dkimpy_milter/dnsplug.py

@@ -84,7 +84,7 @@ class Session(object):
             raise DNSError('Length of CNAME chain exceeds %d' % MAX_CNAME)
         cnames[name] = cname
         if cname in cnames:
-            raise DNSError, 'CNAME loop'
+            raise DNSError('CNAME loop')
         result = self.dns(cname, qtype, cnames=cnames)
     return result
 
@@ -103,16 +103,16 @@ def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
         #
         if resp.header['tc'] == True:
           if not tcpfallback:
-              raise DNS.DNSError, 'DNS: Truncated UDP Reply, SPF records should fit in a UDP packet'
+              raise DNS.DNSError('DNS: Truncated UDP Reply, SPF records should fit in a UDP packet')
           try:
               req = DNS.DnsRequest(name, qtype=qtype, protocol='tcp',
                         timeout=timeout)
               resp = req.req()
-          except DNS.DNSError, x:
+          except DNS.DNSError as x:
-              raise DNS.DNSError, 'TCP Fallback error: ' + str(x)
+              raise DNS.DNSError('TCP Fallback error: ' + str(x))
         return [((a['name'], a['typename']), a['data']) for a in resp.answers]
-    except IOError, x:
+    except IOError as x:
-        raise DNS.DNSError, 'DNS: ' + str(x)
+        raise DNS.DNSError('DNS: ' + str(x))
 
 def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=30):
   retVal = []
@@ -164,5 +164,5 @@ if __name__ == '__main__':
   import sys
   s = Session()
   for n,t in zip(*[iter(sys.argv[1:])]*2):
-    print n,t
+    print(n,t)
-    print s.dns(n,t)
+    print(s.dns(n,t))

dkimpy_milter/util.py

@@ -115,45 +115,59 @@ def write_pid(milterconfig):
     """Write PID in pidfile.  Will not overwrite an existing file."""
     import os
     import syslog
-    if not os.path.isfile(milterconfig.get('PidFile')):
+    pidfile = milterconfig.get('PidFile')
+    if pidfile is None:
+        return
+    if not os.path.isfile(pidfile):
         pid = str(os.getpid())
         try:
-            f = open(milterconfig.get('PidFile'), 'w')
+            f = open(pidfile, 'w')
         except IOError as e:
             if str(e)[:35] == '[Errno 2] No such file or directory':
-                piddir = milterconfig.get('PidFile').rsplit('/', 1)[0]
+                piddir = pidfile.rsplit('/', 1)[0]
                 os.mkdir(piddir)
                 user, group = user_group(milterconfig.get('UserID'))
                 os.chown(piddir, user, group)
-                f = open(milterconfig.get('PidFile'), 'w')
+                f = open(pidfile, 'w')
                 if milterconfig.get('Syslog'):
                     syslog.syslog('PID dir created: {0}'.format(piddir))
             else:
                 if milterconfig.get('Syslog'):
                     syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'
-                                  .format(milterconfig.get('PidFile'), e))
+                                  .format(pidfile, e))
                 raise
         f.write(pid)
         f.close()
         user, group = user_group(milterconfig.get('UserID'))
-        os.chown(milterconfig.get('PidFile'), user, group)
+        os.chown(pidfile, user, group)
     else:
         if milterconfig.get('Syslog'):
             syslog.syslog('Unable to write pidfle {0}.  File exists.'
-                          .format(milterconfig.get('PidFile')))
+                          .format(pidfile))
         raise RuntimeError('Unable to write pidfle {0}.  File exists.'
-                           .format(milterconfig.get('PidFile')))
+                           .format(pidfile))
     return pid
 
 
-def own_socketfile(milterconfig):
+def own_socketfile(milterconfig, sockname=None):
     """If socket is Unix socket, chown to UserID before dropping privileges"""
     import os
     user, group = user_group(milterconfig.get('UserID'))
-    if milterconfig.get('Socket')[:1] == '/':
+    offset = None
-        os.chown(milterconfig.get('Socket')[1:], user, group)
+    if sockname is None:
-    if milterconfig.get('Socket')[:6] == "local:":
+        sockname = milterconfig.get('Socket')
-        os.chown(milterconfig.get('Socket')[6:], user, group)
+    if sockname is None:
+        return
+    if sockname[:1] == '/':
+        offset = 0
+    elif sockname[:6] == "local:":
+        offset = 6
+    elif sockname[:5] == "unix:":
+        offset = 5
+
+    if offset is not None:
+        if os.path.exists(sockname[offset:]):
+            os.chown(sockname[offset:], user, group)
 
 
 def read_keyfile(milterconfig, keytype):
@@ -176,3 +190,26 @@ def read_keyfile(milterconfig, keytype):
     for line in keylist:
         key += line
     return key
+
+def read_keytable(milterconfig, tabletype):
+    """Read keytables into in memory configuration data so all keys are read
+    before priviledges are dropped."""
+    import syslog
+    if tabletype == "RSA":
+        tablefile = milterconfig.get('KeyTable')
+    if tabletype == "Ed25519":
+        tablefile = milterconfig.get('KeyTableEd25519')
+    if milterconfig.get(tablefile):
+        keytabledata = []
+        try:
+            f = open(milterconfig.get(tablefile))
+            for row in f:
+                keytabledata.append(row) 
+            f.close()
+        except IOError as e:
+            if milterconfig.get('Syslog'):
+                syslog.syslog('Unable to read keytable {0}.  IOError: {1}'
+                              .format(tablefile, e))
+            raise
+        
+    return keytabledata

etc/dkimpy-milter.conf

@@ -38,7 +38,7 @@ Socket			inet:8892@localhost
 ###  Name of the file where the filter should write its pid before beginning
 ###  normal operations.
 #
-PidFile			/var/run/dkimpy-milter/dkimpy-milter.pid
+PidFile			/run/dkimpy-milter/dkimpy-milter.pid
 
 ##  Userid userid
 ###      default dkimpy-milter

man/dkimpy-milter.8

@@ -127,7 +127,6 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.IX Title "dkimpy-milter 8"
 .TH dkimpyy-milter 8
 .SH NAME
 .B dkimpy
@@ -269,7 +268,7 @@ proposal, and Cisco's
 .B Internet Identified Mail
 (IIM) proposal.
 .SH VERSION
-This man page covers version 0.9.4 of
+This man page covers version 1.1.0 of
 .I dkimpy-milter.
 .SH COPYRIGHT
 Copyright (c) 2005-2008, Sendmail, Inc. and its suppliers.  All rights
@@ -278,7 +277,7 @@ reserved.
 Copyright (c) 2009-2013, 2015, The Trusted Domain Project.
 All rights reserved.
 
-Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>
+Copyright (c) 2018, 2019 Scott Kitterman <scott@kitterman.com>
 .SH SEE ALSO
 .I dkimpy-milter.conf(5), sendmail(8)
 .P
@@ -292,4 +291,6 @@ RFC6376 - DomainKeys Identified Mail
 .P
 RFC7601 - Message Header Field for Indicating Message Authentication Status
 .P
-draft-ietf-dcrup-dkim-crypto - A new cryptographic signature method for DKIM
+RFC8301 - Cryptographic Algorithm and Key Usage Update to DomainKeys Identified Mail (DKIM)
+.P
+RFC8463 - A New Cryptographic Signature Method for DomainKeys Identified Mail (DKIM)

man/dkimpy-milter.conf.5

@@ -127,16 +127,13 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.IX Title "dkimpy-milter.conf 5"
 .TH dkimpy-milter.conf 5 "2018-02-12"
 .SH "NAME"
 dkimpy-milter \- Python milter for DKIM signing and validation
 .SH "VERSION"
-.IX Header "VERSION"
+1\.1\.0
-0\.9\.2
 
 .SH "DESCRIPTION"
-.IX Header "DESCRIPTION"
 .I dkimpy-milter(8)
 implements the
 .B DKIM
@@ -160,18 +157,15 @@ The provided setup.py installs this configuration file in /etc or
 Command line invocation of parameters as is done by OpenDKIM is not supported.
 
 .SH "USAGE"
-.IX Header "USAGE"
 Usage:
   dkimpy-milter [/etc/dkimpy-milter.conf]
 
 .SH "OTHER DOCUMENTATION"
-.IX Header "OTHER DOCUMENTATION"
 This documentation assumes you have read Postfix's README_FILES/MILTER_README
 (or Sendmail equivalent) and are generally familiar with Domain Keys Identified
 Mail (DKIM).  See RFC 6376 for details.
 
 .SH "SYNOPSIS"
-.IX Header "SYNOPSIS"
 
 dkimpy-milter operates with a default installed configuration file and 
 set of default configuration options that are used if the configuration file
@@ -181,14 +175,12 @@ files can be used directly.  Not all OpenDKIM options are supported.  If an
 unsupported option from OpenDKIM is specified, an error will be raised.
 
 .SH "DESCRIPTION"
-.IX Header "DESCRIPTION"
 
 Configuration options are described here and in the configuration file 
 provided with the package.  The provided setup.py installs this configuration 
 file in /etc or /usr/local/etc.
 
 .SH "OPTIONS"
-.IX Header "OPTIONS"
 
 .TP
 .I AuthservID (string)
@@ -229,12 +221,16 @@ domains will be verified rather than being signed.
 
 This parameter is not required if a
 .I SigningTable
+or
+.I SigningTableEd25519
 is in use; in that case, the list of signed domains is implied by the
-lines in that file. [SigningTable NOT IMPLEMENTED]
+lines in that file.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+or
+.I KeyTableD25119
+is defined.
 
 .TP
 .I InternalHosts (dataset)
@@ -252,7 +248,7 @@ address explicitly. [PeerList NOT IMPLEMENTED]
 Gives the location of a PEM-formatted private key to be used for RSA signing
 all messages.  Ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
 .I KeyFileEd25519 (string)
@@ -260,7 +256,17 @@ Gives the location of a Ed25519 private key to be used for Ed25519 signing
 all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
 generation (as used in dkimpy).  Ignored if a 
 .I KeyTableEd25519
-is defined.  [KeyTableEd25519 NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: direct specification of keys in the table as is done by OpenDKIM is not supported.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: direct specification of keys in the table as is done by OpenDKIM is not support
+ed.
+
 
 .TP
 .I MacroList (dataset)
@@ -317,7 +323,13 @@ When signing mode is enabled, one of the following combinations must also
 be set:
 (a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
 (b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
-[fooTable options NOT IMPLEMENTED]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
 
 .TP
 .I PeerList (dataset)
@@ -339,7 +351,7 @@ will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
 .TP
 .I PidFile (string)
 Specifies the path to a file that should be created at process start
-containing the process ID.
+containing the process ID.  If not specified, no such file will be created.
 
 .TP
 .I Selector (string)
@@ -353,7 +365,7 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
 .I SelectorEd25519 (string)
@@ -367,7 +379,33 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTableEd25519
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select one or more signatures to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field. Entries are checked in the order in which they appear in the file. ["refile support not implemented"].
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+.TP
+.I SigningTableEd25519 (dataset)
+
+Defines a table used to select one or more signatures to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field. Entries are checked in the order in which they appear in the file. ["refile support not implemented"].
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
 
 .TP
 .I Socket (string)
@@ -392,6 +430,12 @@ is not given as either a hostname or an IP address, the socket will be
 listening on all interfaces.  A literal IP address must be enclosed in
 square brackets.  This option is mandatory in the configuration file.
 
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
 .TP
 .I Syslog (Boolean)
 Log via calls to
@@ -441,7 +485,6 @@ unless an alternate
 is specified.
 
 .SH "AUTHORS"
-.IX Header "AUTHORS"
 \ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
 It is based on dkimpy-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
 Copyright (c) 2013-2015 Stuart D. Gathman

setup.py

@@ -1,7 +1,7 @@
-#! /usr/bin/python
+#! /usr/bin/python3
 # dkimpy-milter: A DKIM signing/verification Milter application
 # Author: Scott Kitterman <scott@kitterman.com>
-# Copyright 2018 Scott Kitterman
+# Copyright 2018,2019 Scott Kitterman
 """    This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; either version 2 of the License, or
@@ -18,33 +18,32 @@
 
 from setuptools import setup
 import os
-import dkimpy_milter
 
 description = "Domain Keys Identified Mail (DKIM) signing/verifying milter for Postfix/Sendmail."
 
 kw = {}  # Work-around for lack of 'or' requires in setuptools.
 try:
-    import DNS
+    import dns
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'PyDNS']
+    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'dnspython']
 except ImportError:  # If PyDNS is not installed, prefer dnspython
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'ipaddress', 'dnspython']
+    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'Py3DNS']
 
 setup(
     name='dkimpy-milter',
-    version=dkimpy_milter.__version__,
+    version='1.1.2',
     author='Scott Kitterman',
     author_email='scott@kitterman.com',
     url='https://launchpad.net/dkimpy-milter',
     description=description,
     download_url = "https://pypi.python.org/pypi/dkimpy-milter",
     classifiers= [
-        'Development Status :: 4 - Beta',
+        'Development Status :: 5 - Production/Stable',
         'Environment :: No Input/Output (Daemon)',
         'Intended Audience :: System Administrators',
         'License :: OSI Approved :: GNU General Public License (GPL)',
         'Natural Language :: English',
         'Operating System :: POSIX',
-        'Programming Language :: Python :: 2 :: Only',
+        'Programming Language :: Python :: 3 :: Only',
         'Topic :: Communications :: Email :: Mail Transport Agents',
         'Topic :: Communications :: Email :: Filters',
         'Topic :: Security',
@@ -59,8 +58,8 @@ setup(
     data_files=[(os.path.join('share', 'man', 'man5'),
         ['man/dkimpy-milter.conf.5']), (os.path.join('share', 'man', 'man8'),
         ['man/dkimpy-milter.8']), ('etc', ['etc/dkimpy-milter.conf']),
-        (os.path.join('/lib', 'systemd', 'system'),
+        (os.path.join('lib', 'systemd', 'system'),
-        ['system/dkimpy-milter.service']),(os.path.join('/etc', 'init.d'),
+        ['system/dkimpy-milter.service']),(os.path.join('etc', 'init.d'),
         ['system/dkimpy-milter'])],
     zip_safe = False,
     **kw

system/dkimpy-milter

@@ -13,8 +13,6 @@
 # Provides:          dkim-milter dkim-milter-python dkimpy-milter
 # Required-Start:    $remote_fs $syslog $network $time
 # Required-Stop:     $remote_fs $syslog $network
-# Should-Start:
-# Should-Stop:
 # Default-Start:     2 3 4 5
 # Default-Stop:      0 1 6
 # Short-Description: dkimpy-milter
@@ -24,14 +22,14 @@ prefix="/usr/local"
 exec_prefix=${prefix}
 sysconfdir="/etc/dkimpy-milter"
 bindir="${exec_prefix}/bin/"
-RUNDIR="/var/run/dkimpy-milter"
+RUNDIR="/run/dkimpy-milter"
 DAEMON=${bindir}/dkimpy-milter
 PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
 NAME=dkimpy-milter
 DESC="Python DKIM Milter"
 USER=dkimpy-milter
 GROUP=dkimpy-milter
-SOCKET=$RUNDIR/dkimpy-milter.pid
+SOCKET=$RUNDIR/dkimpy-milter.sock
 
 test -x $DAEMON || exit 0
 
@@ -70,8 +68,8 @@ case "$1" in
 		fi
 	fi
 
-	start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+        start-stop-daemon --start --quiet --pidfile $RUNDIR/$NAME.pid --startas \
-		$RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+                $DAEMON $sysconfdir/$NAME.conf --name $NAME --test > /dev/null \
 	echo "$NAME."
 	;;
   stop)

system/dkimpy-milter.service

@@ -1,11 +1,12 @@
 [Unit]
 Description=DKIMpy Milter
-After=syslog.target network.target
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5) 
+After=network.target
 
 [Service]
 Type=simple
-PIDFile=/var/run/dkimpy-milter/dkimpy-milter.pid
+PIDFile=/run/dkimpy-milter/dkimpy-milter.pid
-ExecStart=/usr/local/bin/dkimpy-milter 
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter.conf 
 
 [Install]
 WantedBy=multi-user.target

system/socket-activation/README.md

@@ -0,0 +1,32 @@
+This directory contains example systemd unit files for running a
+supervised, socket-activated instance of dkimpy-milter.
+
+There are several advantages of using socket activation:
+
+- dkimpy-milter never runs with elevated privileges, they are dropped
+  before any dkimpy-milter code is executed.
+   
+- The socket is opened before dkimpy-milter runs.  This means that
+  clients can connect() to the socket immediately.  So even if there
+  is a delay in dkimpy-milter startup, or in libmilter itself, the
+  connection will not fail.
+  
+- You can set the privileges of a listening Unix-domain socket by an
+  override of ListenGroup= in dkimpy-milter.socket (see
+  systemd.unit(5) for how to override).  This lets you control who has
+  access to the daemon with finer granularity than is available with
+  dkimpy-milter on its own.
+  
+- dkimpy-milter will not consume system resources if it is not used.
+
+- A fully-supervised dkimpy-milter needs no PIDFile, UMask, UserID, or
+  Socket configuation.  This eliminates common race conditions and
+  startup failures, and simplifies the resulting configuration file.
+  
+There is one downside to using socket activation:
+
+- it will only work on systems where libmilter can support connection
+  strings like "fd:3".  This has been supported on Debian and derived
+  systems since sendmail 8.14.4-6 (before Debian Jessie, in early
+  2014), see for example:
+  https://sources.debian.org/src/sendmail/8.15.2-8/debian/patches/socket_activation.patch/

system/socket-activation/dkimpy-milter.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=/usr/bin/dkimpy-milter /etc/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.socket

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=/run/dkimpy-milter/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

tests/00_minimal.miltertest

@@ -0,0 +1,12 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." complete")
+   end
+end

tests/01_connect.miltertest

@@ -0,0 +1,40 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+         error("mt.conninfo() failed "..keytype.." "..func)
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.conninfo() unexpected reply "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_ADDHDRS) then
+         print("could add headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not add headers "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_CHGHDRS) then
+         print("could change headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not change headers "..keytype.." "..func)
+      end
+
+-- -- FIXME: this part of the test fails, as apparently the
+-- -- dkimpy-milter claims the right to change the body of a message,
+-- -- even though it shouldn't.  How can we fix the negotiation?
+--      if mt.test_action(conn, SMFIF_CHGBODY) then
+--         error("mt.test_action() says could change body "..keytype.." "..func)
+--      else
+--         print("could not change body "..keytype.." "..func)
+--      end
+
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." test complete")
+   end
+end

tests/02_sign_message.miltertest

@@ -0,0 +1,100 @@
+-- -*- lua -*-
+
+msg = {
+   ['headers'] = {
+      ['From'] = 'Alice <alice@example.net>',
+      ['Message-Id'] = '<dkimpy-milter-test-02@example.net>',
+      ['To'] = 'Bob <bob@example.biz>',
+      ['Date'] = 'Mon, 18 Feb 2019 08:32:50 -0500',
+      ['Subject'] = 'Signing test',
+      ['Content-Type'] = 'text/plain',
+   },
+   ['body'] = "This is a test!\r\n",
+}
+
+-- returns miltertest connection object
+function connect_and_send (sockname, headers, body)
+   conn = mt.connect(sockname)
+   if conn == nil then
+      error "mt.connect() failed"
+   end
+   if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+      error "mt.conninfo() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.conninfo() unexpected reply"
+   end
+
+   -- mt.macro(conn, SMFIC_MAIL, "i", "simple-message")
+   if mt.mailfrom(conn, "<alice@example.net>") ~= nil then
+      error "mt.mailfrom() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.mailfrom() unexpected reply"
+   end
+   -- mt.rcptto() is called implicitly
+
+   -- send headers
+   for key,value in pairs(headers) do
+      if mt.header(conn, key, value) ~= nil then
+         error("mt.header(" .. key .. ") failed")
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.header(" .. key .. ") unexpected reply")
+      end
+   end
+   -- send EOH
+   if mt.eoh(conn) ~= nil then
+      error "mt.eoh() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.eoh() unexpected reply"
+   end
+
+   -- send body
+   if mt.bodystring(conn, body) ~= nil then
+      error "mt.bodystring() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.bodystring() unexpected reply"
+   end
+   -- end of message; let the filter react
+   if mt.eom(conn) ~= nil then
+      error "mt.eom() failed"
+   end
+   reply = mt.getreply(conn)
+   if reply ~= SMFIR_CONTINUE then
+      error ("mt.eom() unexpected reply: " .. reply)
+   end
+   return conn
+end
+
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   mt.echo("testing "..keytype)
+   signing = connect_and_send("unix:"..keytype..".signing.sock", msg.headers, msg.body)
+   -- verify that a test header field got added
+   if not mt.eom_check(signing, MT_HDRINSERT) then
+      error "no header added by signer"
+   end
+
+   signature = mt.getheader(signing, "DKIM-Signature", 0)
+
+   mt.disconnect(signing)
+
+   mt.echo("DKIM-Signature: " .. signature)
+
+   msg.headers['DKIM-Signature'] = signature
+
+   verify = connect_and_send("unix:"..keytype..".verify.sock", msg.headers, msg.body)
+
+   if not mt.eom_check(verify, MT_HDRINSERT) then
+      error "no header added in verify"
+   end
+
+   authres = mt.getheader(verify, "Authentication-Results", 0)
+   mt.echo("Authentication-Results: "..authres)
+
+   mt.disconnect(verify)
+
+   mt.echo(keytype.." complete")
+end

tests/dkimpy-milter

@@ -0,0 +1,2 @@
+#!/bin/sh
+python3 -m dkimpy_milter "$@"

tests/runtests

@@ -0,0 +1,84 @@
+#!/bin/bash
+
+set -e
+WORKDIR=$(mktemp -d)
+TESTDIR=$(realpath "$(dirname "$0")")
+DKIMPY_MILTER=${DKIMPY_MILTER:-"$TESTDIR/dkimpy-milter"}
+KEY_TYPES=(ed25519 rsa)
+
+cd "$WORKDIR"
+
+printf "Testing %s from directory %s\n" "$DKIMPY_MILTER" "$WORKDIR"
+
+for keytype in "${KEY_TYPES[@]}"; do
+    dknewkey --ktype "$keytype" "testkey.$keytype"
+    if [ "$keytype" = ed25519 ]; then
+        keyfile=KeyFileEd25519
+        selector=SelectorEd25519
+    else
+        keyfile=KeyFile
+        selector=Selector
+    fi
+    cat > "$keytype.signing.conf" <<EOF
+Domain example.net
+$keyfile testkey.$keytype.key
+$selector testkey
+Socket unix:$keytype.signing.sock
+PidFile $keytype.signing.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.verify.conf" <<EOF
+Socket unix:$keytype.verify.sock
+PidFile $keytype.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+done
+
+cleanup() {
+    echo cleaning up jobs:
+    jobs
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify; do
+            if [ -s "$keytype.$func.pid" ] && kill -0 "$(cat "$keytype.$func.pid")"; then
+                kill "$(cat $keytype.$func.pid)"
+            fi
+        done
+    done
+    wait
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify; do
+            errdata="$keytype.$func.stderr"
+            if [ -s "$errdata" ]; then
+                printf -- "-> %s:\n" "$errdata"
+                cat "$errdata"
+                printf -- "-> end %s\n" "$errdata"
+            fi
+        done
+    done
+    rm -rf "$WORKDIR"
+}
+
+for keytype in "${KEY_TYPES[@]}"; do
+    for func in signing verify; do
+        PYTHONPATH="$(dirname "$TESTDIR")" "$DKIMPY_MILTER" "$keytype.$func.conf" 2>"$keytype.$func.stderr" &
+    done
+done
+trap cleanup EXIT
+
+# ugly ugly (how are we supposed to know that the milters are all ready?):
+sleep 2
+
+# uses miltertest from opendkim:
+for x in ${TESTS:-"$TESTDIR"/*.miltertest}; do
+    if ! [ -e "$x" ]; then
+        if [ -e "$TESTDIR/$x" ]; then
+            x="$TESTDIR/$x"
+        fi
+    fi
+    printf -- "-> running %s...\n" "$x"
+    miltertest -s "$x"
+done