Logging fixups: Don't traceback for non-UTF-8 data in mail headers and don't put byte string markers in logs (some remain, but are from dkimpy and should be fixed there), related to LP: #1980821

Clarify that key names from SigningTable apply to RSA and ed25519 keys.
The way to sign with both algorithms is to provide keys in both
KeyTables, with the same name.

CHANGES

@@ -1,3 +1,14 @@
+1.2.3
+ - Logging fixups: Don't traceback for non-UTF-8 data in mail headers and
+   don't put byte string markers in logs (some remain, but are from dkimpy
+   and should be fixed there), related to LP: #1980821.
+
+1.2.2 2020-08-09
+ - Improve README.md formating for markdown display on pypi
+ - Improve documentation in dkimpy-milter.conf (5) and README.md for signing
+   for multiple domains (Thanks to Stefano Rivera)
+ - Minimal fix for dnspython 2.0.0 compatibility (still works with 1.16.0)
+
 1.2.1 2020-01-04
  - Fix expand option not to fail if files are missing since socket activation
    service files are not shipped in the sdist

README.md

@@ -52,6 +52,7 @@ The package includes a custom setup command called expand.  It allows various
 file locations in init scripts, man pages, and config files to be over-ridden
 at install time.
 
+
     expand: Expand @@ variables in input files, simlar to make macros.
     user_options:
       --sysconfigdir=, e: Specify system configuration directory.
@@ -135,9 +136,9 @@ for the above might look like this:
 	comkey	example.com:bar:/usr/local/etc/dkim/keys/excom
 	netkey	example.net:baz:/usr/local/etc/dkim/keys/exnet
 
-If also signing with ed25519, specify a KeyTableEd25519 pointing to the keys
-needed for ed25519.  Both KeyTable and KeyTableEd25519 are evaluated if there
-is a SigningTable (see below).
+If also signing with ed25519, specify a KeyTableEd25519, with the same
+names, pointing to the keys needed for ed25519.  Both KeyTable and
+KeyTableEd25519 are evaluated if there is a SigningTable (see below).
 
 Per the documentation, multi-field data sets that are made of flat files have
 the fields separated by colons, but the key and value(s) are separated by
@@ -237,7 +238,7 @@ and deserve consideration.
 
     By default, sendmail quotes to address header fields when there are no
     quotes and the display part of the address contains a period or an
-    apostrophe.  However, opendkim only sees the raw, unmodified form of
+    apostrophe.  However, dkimpy-milter only sees the raw, unmodified form of
     the header field, and so the content that gets verified and what gets
     signed will not be the same, guaranteeing the attached signature is not
     valid.
@@ -263,16 +264,16 @@ and deserve consideration.
     To: very long name <a@example.org>,
     	anotherloo...ong name b <b@example.org>
 
-    This rewrite is also done after opendkim has seen the message, meaning
-    the signature opendkim attaches to the message does not match the
-    content it signed.  There is not a known configuration change to
+    This rewrite is also done after dkimpy-milter has seen the message,
+    meaning the signature dkimpy-milter attaches to the message does not match
+    the content it signed.  There is not a known configuration change to
     mitigate this mutation.
 
     The only known mechanism for dealing with this is to have distinct
-    instances of opendkim do the verifying (inbound) and signing (outbound)
-    so that the version that arrives at the signing instance is already
-    in the rewritten form, guaranteeing the input and output are the same
-    and thus the signature matches the payload.
+    instances of dkimpy-milter do the verifying (inbound) and signing
+    (outbound) so that the version that arrives at the signing instance is
+    already in the rewritten form, guaranteeing the input and output are the
+    same and thus the signature matches the payload.
 
 ### POSTFIX
 

dkimpy_milter/__init__.py

@@ -107,9 +107,10 @@ class dkimMilter(Milter.Base):
     # envfrom (MAIL FROM in the SMTP protocol) seems to mark the start
     # of each message.
     @Milter.noreply
-    def envfrom(self, f, *str):
+    def envfrom(self, f, *stri):
         if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 2:
-            syslog.syslog("mail from: {0} {1}".format(f, str))
+            f = str(bytes(f, encoding='utf-8', errors='replace'))[2:-1]
+            syslog.syslog("mail from: {0} {1}".format(f, stri))
         self.fp = io.BytesIO()
         self.mailfrom = f
         t = parse_addr(f)
@@ -133,7 +134,8 @@ class dkimMilter(Milter.Base):
         if lname == 'dkim-signature':
             if (self.conf.get('Syslog') and
                     self.conf.get('debugLevel') >= 1):
-                syslog.syslog("{0}: {1}".format(name, val))
+                val2 = str(bytes(val, encoding='utf-8', errors='replace'))[2:-1]
+                syslog.syslog("{0}: {1}".format(name, val2))
             self.has_dkim += 1
         if lname == 'from':
             fname, self.author = parseaddr(val)
@@ -143,7 +145,8 @@ class dkimMilter(Milter.Base):
                 pass # self.author was not a proper email address
             if (self.conf.get('Syslog') and
                     self.conf.get('debugLevel') >= 1):
-                syslog.syslog("{0}: {1}".format(name, val))
+                val2 = str(bytes(val, encoding='utf-8', errors='replace'))[2:-1]
+                syslog.syslog("{0}: {1}".format(name, val2))
         elif lname == 'authentication-results':
             self.arheaders.append(val)
         if self.fp:
@@ -181,7 +184,8 @@ class dkimMilter(Milter.Base):
                     self.chgheader('authentication-results', i, '')
                     if (self.conf.get('Syslog') and
                             self.conf.get('debugLevel') >= 1):
-                        syslog.syslog('REMOVE: {0}'.format(val))
+                        val2 = str(bytes(val, encoding='utf-8', errors='replace'))[2:-1]
+                        syslog.syslog('REMOVE: {0}'.format(val2))
             except:
                 # Don't error out on unparseable AR header fiels
                 pass
@@ -363,8 +367,12 @@ class dkimMilter(Milter.Base):
             try:
                 dnsoverride = self.conf.get('DNSOverride')
                 if isinstance(dnsoverride, str):
+                    timeout = 5
+                    domain = self.fdomain
+                    def dnsfunc(domain, timeout=timeout, dnsoverride=dnsoverride):
+                        return dnsoverride
                     syslog.syslog("DNSOverride: {0}".format(dnsoverride))
-                    res = d.verify(idx=y, dnsfunc=lambda _x: dnsoverride)
+                    res = d.verify(idx=y, dnsfunc=dnsfunc)
                 else:
                     res = d.verify(idx=y)
                 algo = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
@@ -424,7 +432,8 @@ class dkimMilter(Milter.Base):
                     if self.conf.get('Syslog'):
                         if d.domain:
                             syslog.syslog('DKIM: Fail ({0})'
-                                          .format(d.domain.lower()))
+                                .format(str(d.domain.lower(), 'UTF-8',
+                                errors='replace')))
                         else:
                             syslog.syslog('DKIM: Fail, unextractable domain')
             if res:
@@ -434,7 +443,7 @@ class dkimMilter(Milter.Base):
             res = False
             if self.header_d:
                 self.arresults.append(
-                    authres.DKIMAuthenticationResult(result=result,
+                    authres.DKIMAuthenticationResult(result=result[2:-1],
                                                  header_i=self.header_i,
                                                  header_d=self.header_d,
                                                  header_a=self.header_a,

dkimpy_milter/dnsplug.py

@@ -127,7 +127,7 @@ def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=5):
       elif qtype == 'PTR':
         retVal.append(((name, qtype), rdata.target.to_text(True)))
       elif qtype == 'TXT' or qtype == 'SPF':
-        retVal.append(((name, qtype), rdata.strings))
+        retVal.append(((name, qtype), list(rdata.strings)))
   except dns.resolver.NoAnswer:
     pass
   except dns.resolver.NXDOMAIN:

man/dkimpy-milter.conf.5.in

@@ -428,7 +428,7 @@ of this field.
 .TP
 .I SigningTable (dataset)
 
-Defines a table used to select one or more signing identities to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+Defines a table used to select a signing identity to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
 
 If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.