A-label

(LP: #1901445)

.gitignore

@@ -1,3 +1,4 @@
 dist
 dkimpy_milter.egg-info
 *.pyc
+*~

CHANGES

@@ -1,3 +1,143 @@
+1.2.3 2023-02-26
+ - Improve support for non-ASCII email messages.  Anything UTF-8 should work
+   (including correct signing/verification).  For messages that contain header
+   fields with non-ASCII or UTF-8 content, signatures are likely fail
+   verification, but the milter should continue to run.  (Thanks to Casper
+   Bruun for help with this)
+ - Set minimum pymilter and dkimpy versions in setup.py to those that will
+   work reliably with non-ASCII content.
+ - Fixed support for percent in KeyTable - Thanks to Mika Tiainen
+ - Fix formatting for MinimumKeyBits in dkimpy-milter.conf(5)
+   (Closes: #995335)
+ - Reset the i= signature identity in get_identities_sign() (Closes: #981157)
+ - Improve documentation of inter-relationship between Mode, InternalHosts,
+   MacroList, and MacroListVerify options in dkimpy-milter.conf.5 (Closes:
+   #969215)
+ - Fix subdomain signing with top-level organizational domain (LP: #1999434)
+   - Thanks to Matthias Hunstock for the report and the fix
+ - Fix comma separated list processing in dkimpy_milter/config.py
+   (LP: #1901445)
+
+1.2.2 2020-08-09
+ - Improve README.md formating for markdown display on pypi
+ - Improve documentation in dkimpy-milter.conf (5) and README.md for signing
+   for multiple domains (Thanks to Stefano Rivera)
+ - Minimal fix for dnspython 2.0.0 compatibility (still works with 1.16.0)
+
+1.2.1 2020-01-04
+ - Fix expand option not to fail if files are missing since socket activation
+   service files are not shipped in the sdist
+ - Correct dkimpy-milter.conf file install location to match expand locations
+
+1.2.0 2020-01-03
+ - Add support for SigningTable, KeyTable, and KeyTableEd25519 (LP: #1797397)
+ - Add support for specifying MinimumKeyBits for RSA signatures
+ - Add support for SignHeaders feature, thanks to Ralph Seichter for the patch
+ - Add support for specifying DNSTimeout (bumps required dkimpy version to 1.0)
+ - Add information on message content conversion to README
+ - Add new expand option to setup.py so various file system locations can be
+   specified at build/install time rather than being hard coded
+ - Install openrc init file for Gentoo and other openrc users
+ - Add support for passing PID file name on command line to make it easier to
+   keep system init and daemon configuration in sync
+ - Add support for storing DKIM failed mails in a specified
+   DiagnosticDirectory
+ - Fix startup logging so it provides information at a useful time
+ - Fix verify processing so missing (optional) i= tag doesn't cause the milter
+   to fail  (LP: #1842250)
+ - Fix message extraction so that signing in the same pass through the milter
+   as verifying works correctly
+ - Add debug logging for content type to assist troubleshooting MIME
+   conversion issues
+ - Fix variable initialization so mailformed mails missing body From do not
+   cause a traceback (LP: #1844161)
+ - Catch more ascii encoding errors to improve resilience against bad data
+   (LP: #1844189)
+ - Fix sysv init so it works (LP: #1839487)
+ - Make error logging more explicit to aid debugging
+ - Remove SigningTableEd25519 from documentation - it was never implemented
+   and a per algorithm signing table turns out not to be needed
+ - Delete own_socketfile to resolve race condition where the permissions
+   change fails on a Unix socket because it hasn't been created yet (libmilter
+   will do this correctly on its own based on umask, the milter doesn't need
+   to do it) (LP: #1849712)
+
+1.1.0 2019-04-12
+ - Add SubDomains option to enable signing for sub-domains (LP: #1811535)
+ - Port to python3 (LP: #1815502)
+ - Add test suite using opendkim miltertest
+ - When Socket is absolute path, do not strip leading /
+ - Handle unix: socket prefix the same as local:
+ - Set up correct AuthservID defaults
+ - config: Reassemble strings sensibly
+ - Consistently prefer dnspython to Py3DNS (LP: #1815558)
+
+1.0.1 2019-02-11
+ - Reorder milter start and dropping privileges so permissions on Unix socket
+   are correct (LP: 1797720)
+ - Make domain checks case insensitive for determining if signing should be
+   done (LP: #1815311)
+ - Add additional Sendmail configuration information to README from OpenDKIM
+   update based on input from Дилян Палаузов (LP: #1801619)
+ - Add information on Ed25519 key creation to README (LP: #1815313)
+
+1.0.0 2018-05-11
+ - Minor documentation updates
+ - Deleted reference to obsolete syslog target in unit file
+
+0.9.7 2018-03-19
+ - Made sysv init executable
+ - Add missing documentation key to system/dkimpy-milter.service
+ - Put version directly in setup.py and do not import dkimpy_milter to ease
+   install via pip
+ - Minor sysv init improvments
+
+0.9.6 2018-03-13
+ - Fixed typo in package installation section of README
+ - Added more to README about first run with systemd
+ - Fixed typo in path for fallback location of the config file if one is not
+   provided
+ - Added protection for malformed From addresses.  If the From does not at
+   least have an '@' in the address, then the signing domain is not extracted
+   and the message will not be signed
+
+0.9.5.1 2018-03-10
+ - Add conf file location to systemd unit file
+ - Fix setup.py install locations so they are installed correctly
+
+0.9.5 2018-03-10
+ - Beta 1 (updated Alpha -> Beta warning in README and trove classifiers)
+ - Added support for MacroList option
+ - Added support for MacroListVerify option
+ - Added example in README to show use of MacroList* to separate inbound and
+   outbound mail streams
+ - Added support for SyslogSuccess option (both signing and verifying)
+ - Rationalized logging to be much less verbose unless SyslogSuccess or
+   debugLevel are set - default is generally start/stop/errors only
+ - Fixed install_requires so either dnspython (preferred if neither is
+   installed) or PyDNS satisfies the install requirements
+ - Updated Authentication Results result comment not to mention key size for
+   ed25519 signatures, since it's irrelevant
+ - Enhanced signature verification logging to provide more useful information
+
+0.9.4 2018-03-09
+ - Create PID directory if it is missing
+ - Fix crash when verifying if domain for signing was not set
+ - Fix header folding to use \n only to align with milter protocol
+   requirements
+ - Added information about creating a dedicated user and PID file directory
+   creation to README
+ - Fixed a bug where dkim fail might be reported as pass when verifying
+   multiple signatures and a previous signature had passed
+ - Make RSA signatures in dkimpy-milter optional, so dkimpy-milter can be
+   added after an existing DKIM signing application to add an Ed25519
+   signature (Thanks to A. Schulze for the patch)
+ - Added support for AuthservID option
+ - Added support for InternalHosts option (ipaddress and either dns (dnspython)
+   or pydns (DNS) modules are now required)
+ - Added support for DiagnosticDirectory and updated dkimpy-milter specifics in
+   dkimpy-milter.conf.5
+
 0.9.3 2018-03-02
  - Fixup csl dataset processing for single item lists
  - file: dataset support

Cheatsheet.txt

@@ -0,0 +1,25 @@
+Welcome to Cheatsheet.txt, licensed under CC-0. No attribution required.
+But it is writen by Diskette (diskette@dailitation.xyz)
+
+Information regarding the [] flags are in dkimpy repository.
+
+[__init__.py]
+Initialization file,
+
+class dkimMilter, a milter for dkim
+
+What are those
+self.fp localpart
+self.fdomain domain part
+self.iequals i still have no idea
+
+def header
+
+
+define check_dkim, I assume that this checks dkim, how?
+
+def sign_dkim
+d = dkim.DKIM(txt)
+
+[dnsplyug.py]
+File for interfacing with DNS

README

@@ -1,41 +0,0 @@
-This is a DKIM signing and verification milter.  In theory it works with both
-Postfix and Sendmail, but the author has zero experience with Sendmail, so
-reports of success/failure with Sendmail and patches are welcom.
-
-The configuration file is designed to be compatible with OpenDKIM, but only
-a subset of OpenDKIM options are supported.  If an unsupported option is
-specified, an error will be raised.
-
-This package includes a default configuration file and man pages.  For those
-to be installed when installing using setup.py, the following incantation is
-required because setuptools developers decided not being able to do this by
-default is a feature:
-
-python setup.py install --single-version-externally-managed --record=/dev/null
-
-For users of Debian Stable (Debian 9, Codename Squueze), all dependencies are
-available in either the main or backports repositories:
-
-[sudo] apt install python-milter python-nacl
-[sudo] apt install -t squeeze-backports python-authres python-dkim
-
-The preferred method of installation is from PyPi using pip:
-
-[sudo] pip install dkimpy_milter
-
-Using pip will cause required packages to be installed via easy_install if they
-have not been previously installed.
-
-Both a systemd unit file and a sysv init file are provided.  Both make
-assumptions about defaults being used, e.g. if a non-standard pidfile name is
-used, they will need to be updated.  The sysv init file is Debian specific and
-untested, since the developers are not using sysv init.  Feedback/patches
-welcome.
-
-The python DKIM library, dkimpy, requires the entire message being signed or
-verified to be in memory, so dkimpy-milter does not write messages out to a temp
-file.  This may impact performance on low-memory systems.
-
-WARNING: This is an alpha grade release to support interoperability testing with
-Ed25519 signatures and basic functionality.  It is known to be incomplete and
-not suitable for general use.

README.md

@@ -0,0 +1,351 @@
+An SMTPUTF8-approved version of dkimpy-milter
+
+Please do note that there might be some mistakes along the way... No warranty is provided!
+
+This implements support for internationalized email address (RFC 8616)
+
+Cheatsheet.txt is a file for my (diskette@dailitation.xyz) personal note taking.
+
+# OVERVIEW
+
+This is a DKIM signing and verification milter.  It has been tested with both
+Postfix and Sendmail.
+
+The configuration file is designed to be compatible with OpenDKIM, but only
+a subset of OpenDKIM options are supported.  If an unsupported option is
+specified, an error will be raised.
+
+
+# INSTALLATION
+
+This package includes a default configuration file and man pages.  For those
+to be installed when installing using setup.py, the following incantation is
+required because setuptools developers decided not being able to do this by
+default is a feature:
+
+[sudo] python3 setup.py install --single-version-externally-managed --record=/dev/null
+
+For users of Debian Stable (Debian 9, Codename Squeeze), all dependencies are
+available in either the main or backports repositories:
+
+[sudo] apt install python3-milter python3-nacl python3-dnspython
+[sudo] apt install -t stretch-backports python3-authres python3-dkim
+
+It is also available in the Debian package archive:
+
+[sudo] apt install dkimpy-milter [Debian 10 or later]
+[sudo] apt install -t stretch-backports dkimpy-milter [Debian 9]
+
+When installing using the Debian package, all dependencies are automatically
+installed.
+
+The preferred method of installation is from PyPi using pip (if distribution
+packages are not available):
+
+[sudo] pip install dkimpy_milter
+
+Using pip will cause required packages to be installed via easy_install if they
+have not been previously installed.  Because pymilter and PyNaCl are compiled
+Python extensions, the system will need appropriate development packages and
+an C compiler.  Alternately, install these dependencies from distribution/OS
+packages and then pip install dkimpy_milter.
+
+The milter will work with either py3dns (DNS) or dnspython (dns), preferring
+dnspython if both are available.  The dkimpy DKIM module also works with
+either.
+
+## NON-STANDARD INSTALLATION PATHS
+
+The package includes a custom setup command called expand.  It allows various
+file locations in init scripts, man pages, and config files to be over-ridden
+at install time.
+
+
+    expand: Expand @@ variables in input files, simlar to make macros.
+    user_options:
+      --sysconfigdir=, e: Specify system configuration directory.
+      --sbindir=, s: Specify system binary directory [not used].
+      --bindir=, b: Specify binary directory.
+      --rundir=,r: Specify run state directory.
+
+As an example, to change the run directory to /var/run, one would do:
+
+    python3 setup.py expand --rundir=/var/run
+    [sudo] python3 setup.py install --single-version-externally-managed \
+                                --record=/dev/null
+
+or in a single step (the order matters):
+
+    [sudo] python3 setup.py expand --rundir=/var/run install \
+                               --single-version-externally-managed \
+                               --record=/dev/null
+
+# SETUP
+
+## SIGNING KEYS
+
+In order to create DKIM signatures, a private key must be available.  Signing
+keys should be protected (owned by root:root with permissions 600 in a
+directory that is not world readable).  Different keys are required for RSA
+and (if used) Ed25519.
+
+### RSA
+
+Both public and private keys for RSA have standard formats and there are many
+tools available to create them.  Keys must (RFC 8302) have a minimum size of
+1024 bits and should have a size of at least 2048 bits.  The dknewkey script
+that is provided with dkimpy is one such tool:
+
+    dknewkey exampleprivkey
+
+will produce both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  RSA is the default key
+type.  2048 bits is the default key size.
+
+### ED25519
+
+There is no standardized non-binary representation for Ed25519 private keys,
+so in order to generate Ed25519 keys for dkimpy-milter, dkimpy specific tools
+must be used to be compatible.  The same dknewkey script support Ed25519:
+
+    dknewkey --ktype ed25519 anothernewkey
+
+will provide both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  Ed25519 keys do not have
+variable bit lengths.
+
+### COMPLEX SIGNING CONFIGURATIONS
+
+The KeyTable, KeyTableEd25519, and SigningTable are used to define signing
+instructions to the filter where use of Domain, Selector and KeyFile together
+are insufficient.
+
+First, select the type of database you will use for each.  They need not
+be the same.  The "DATA SETS" portion of the dkimpy-milter(8) man page
+describes the possibilities and how they are formatted.  Then, construct those
+databases.
+
+Let's suppose you want to sign for two domains, example.com and example.net.
+Within example.com, you want to sign for user "president" differently than
+everyone else.  Let's say further that you want to use a flat text file.
+
+You've generated private key files for each of these and stored them
+in the directory /usr/local/etc/dkim/keys as files "president", "excom" and
+"exnet", with the obvious intents.  You want to use selectors "foo", "bar"
+and "baz" for those, respectively.  The signing domains match the senders
+(i.e. the signatures for example.com's stuff will be held by example.com,
+and example.net likewise).
+
+First, write the KeyTable.  This is a list of the keys you intend to use,
+and you just assign arbitrary names to them.  So as a flat file, the KeyTable
+for the above might look like this:
+
+	preskey	example.com:foo:/usr/local/etc/dkim/keys/president
+	comkey	example.com:bar:/usr/local/etc/dkim/keys/excom
+	netkey	example.net:baz:/usr/local/etc/dkim/keys/exnet
+
+If also signing with ed25519, specify a KeyTableEd25519, with the same
+names, pointing to the keys needed for ed25519.  Both KeyTable and
+KeyTableEd25519 are evaluated if there is a SigningTable (see below).
+
+Per the documentation, multi-field data sets that are made of flat files have
+the fields separated by colons, but the key and value(s) are separated by
+whitespace.
+
+So now we've named each key file, and specified with which selector and domain
+each will be used, and then given each of those groupings a name.  This
+is your KeyTable.  Let's say you put it in /usr/local/etc/dkim/keytable.
+
+Next, write the SigningTable.  This maps senders (by default, taken from the
+From: header field of a message passing through the filter) to which keys
+will be used to sign their mail.  Wildcards are allowed.  So to do what was
+described above, we write it as follows:
+
+	president@example.com	preskey
+	*@example.com		comkey
+	*@example.net		netkey
+
+Since we want to use wildcards, we can't actually use a regular flat file.
+Wildcards require a regular expression file, or "refile".  The above is
+valid format for one of those.  Let's say you put this in
+/usr/local/etc/dkim/signingtable.
+
+Finally, tell the filter that it should use these files by adding this to
+your configuration file:
+
+	KeyTable	/usr/local/etc/dkim/keytable
+	SigningTable	refile:/usr/local/etc/dkim/signingtable
+
+You could put "file:" in front of the filename for the KeyTable just to be
+precise, but "file:" is assumed if the value starts with a "/".
+
+Note: Unlike opendkim, dkimpy-milter will check for "\*" in the signing table
+regardless of if refile is specified or not.  Use of refile is supported for
+compatibility with configurations initially developed for use with opendkim.
+
+## MTA INTEGRATION
+
+Both a systemd unit file and a sysv init file are provided.  Both make
+assumptions about defaults being used, e.g. if a non-standard pidfile name is
+used, they will need to be updated.  The sysv init file uses start-stop-deamon
+from Debian.  It is not portable to systems without that available.
+
+The dkimpy-milter drops priviledges after setup to the user/group specified in
+UserID.  During initial setup, this system user needs to be manually created.
+As an example, using the default dkimpy-user on Debian, the command would be:
+
+    [sudo] adduser --system --no-create-home --quiet --disabled-password \
+               --disabled-login --shell /bin/false --group \
+               --home /run/dkimpy-milter dkimpy-milter
+
+Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
+missing, the milter will create it on startup.
+
+To start dkimpy-milter with systemd for the first time, you will need to take
+the following steps:
+
+    [sudo] systemctl daemon-reload
+    [sudo] systemctl enable dkimpy-milter
+    [sudo] systemctl start dkimpy-milter
+    [sudo] systemctl status dkimpy-milter (to verify it started correctly)
+
+As with all milters, dkimpy-milter needs to be integrated with your MTA of
+choice (Sendmail or Postfix).  When integrating with your MTA, the risk of
+signature invalidation due to content conversion of the message body needs to
+be considered.  See RFC 6376, Section 5.3 for discussion of this issue.  As a
+practical matter, when signing, configure the milter to follow all others that
+might modify the message body.  When verifying, configure the milter before
+other processes that might modify the message body.
+
+### SENDMAIL
+
+Configuration is very similar to opendkim, but needs some adjustment for
+dkimpy-milter. Here's an example configuration line to include in your
+sendmail.mc:
+
+    INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/run/dkimpy-milter/dkimpy-milter.sock')dnl
+
+Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
+and a restart of sendmail.  Note that S= needs to match the value of Socket in
+the dkimpy-milter configuration file.
+
+Milter support should be present by default in most versions of sendmail
+these days, but if not included in your Sendmail build, see:
+http://www.elandsys.com/resources/sendmail/milter.html
+
+#### ISSUES USING SENDMAIL TO SIGN AND VERIFY
+
+When using the sendmail MTA in both signing and verifying mode, there are
+a few issues of which to be aware that might cause operational problems
+and deserve consideration.
+
+(a) When the MTA will be used for relaying emails, e.g. delivering to other
+    hosts using the aliases mechanism, it is important not to break
+    signatures inserted by the original sender.  This is particularly sensitive
+    particular when the sending domain has published a "reject" DMARC policy.
+
+    By default, sendmail quotes to address header fields when there are no
+    quotes and the display part of the address contains a period or an
+    apostrophe.  However, dkimpy-milter only sees the raw, unmodified form of
+    the header field, and so the content that gets verified and what gets
+    signed will not be the same, guaranteeing the attached signature is not
+    valid.
+
+    To direct sendmail not to modify the headers, add this to your sendmail.mc:
+
+    	conf(`confMUST_QUOTE_CHARS', `')
+
+(b) As stated in sendmail's KNOWNBUGS file, sendmail truncates header field
+    values longer than 256 characters, which could mean truncating the domain
+    of a long From: header field value and invalidating the signature.
+    You may wish to consider increasing MAXNAME in sendmail/conf.h to mitigate
+    changing the messages and invalidating their signatures.  This change
+    requires recompiling sendmail.
+
+(c) Similar to (a) above, sendmail may wrap very long single-line recipient
+    fields for presentation purposes; for example:
+
+    To: very long name <a@example.org>,anotherloo...ong name b <b@example.org>
+
+    ...might be rewritten as:
+
+    To: very long name <a@example.org>,
+    	anotherloo...ong name b <b@example.org>
+
+    This rewrite is also done after dkimpy-milter has seen the message,
+    meaning the signature dkimpy-milter attaches to the message does not match
+    the content it signed.  There is not a known configuration change to
+    mitigate this mutation.
+
+    The only known mechanism for dealing with this is to have distinct
+    instances of dkimpy-milter do the verifying (inbound) and signing
+    (outbound) so that the version that arrives at the signing instance is
+    already in the rewritten form, guaranteeing the input and output are the
+    same and thus the signature matches the payload.
+
+### POSTFIX
+
+Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
+README_FILES/MILTER_README).  Here's an example master.cf excerpt that talks
+to two dkimpy-milter instances, one configured for signing and one configured
+for verification:
+
+    smtp       inet  n       -       -       -      -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8892
+        ...
+
+    submission inet  n       -       -       -      -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8891
+        ...
+
+These need to match the Socket value for each dkimpy-milter instance.
+
+Care is required to segregate outbound mail to be signed and inbound mail to
+be verified.  The above example uses two instances of dkimpy-milter to do
+this.  There are many possible ways.  Here is another example using milter
+macros to keep the mail streams segregated:
+
+Postfix master.cf:
+
+    smtp       inet  n       -       -       -       -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8891
+        -o milter_macro_daemon_name=VERIFYING
+        ...
+
+    submission inet n       -       -       -       -       smtpd
+        -o syslog_name=postfix/submission
+        -o smtpd_tls_security_level=encrypt
+        -o smtpd_sasl_auth_enable=yes
+        ...
+        -o milter_macro_daemon_name=ORIGINATING
+        -o smtpd_milters=inet:localhost:8891
+        ...
+
+Dkimpy-milter.conf:
+
+    ...
+    Mode			sv
+    MacroList		daemon_name|ORIGINATING
+    MacroListVerify		daemon_name|VERIFYING
+    ...
+
+
+# NOTES
+
+The python DKIM library, dkimpy, requires the entire message being signed or
+verified to be in memory, so dkimpy-milter does not write messages out to a
+temp file.  This may impact performance on low-memory systems.
+
+DKIM with Ed25519 signatures are described in RFC 8463.  Version 1.0.0 and
+later support Ed25519 signing and verification.  RFC 8301 removed rsa-sha1
+from DKIM.  dkimpy-milter does not sign with rsa-sha1, but still considers
+rsa-sha1 signatures as valid for verification because they are still in
+common use and are not known to be cryptographically broken.
+
+Support for non-ASCII email messages:  Anything UTF-8 should work (including
+correct signing/verification).  For messages that contain header fields with
+non-ASCII or UTF-8 content, signatures are likely fail verification, but the
+milter should continue to run.  RFC 8616 is not supported.

TODO

@@ -25,23 +25,40 @@ Sign based on Domain   implemented      verified
 Canonicalization       implemented      verified
 SyslogFacility         implemented      verified
 
-0.9.5 (Beta)
+0.9.3 (Alpha)
-AuthservID
+File dataset           implemented      verified
-Diagnostics
-DiagnosticDirectory
-InternalHosts
 
-SyslogSuccess
+0.9.4 (Alpha)
+AuthservID      implemented     verified
+DiagnosticDirectory     implemented     verified
+InternalHosts   implemented     verified
+
+0.9.5 (Beta)
+MacroList	implemented	verified
+MacroListVerify	implemented	verified
+SyslogSuccess   implemented     verified
 
 1.0.0
-Convert dkim-milter-python config
+No additional features
-No additional features planned
 
-Plannedataset type support:
+1.0.1
-file://
+Bug fix only, improved documentation
-refile:
+
-db:/.db
+1.1.0
-csl:
+Port to Python 3  implemented	verified
+Subdomain support implemented	verified
+Test suite	  implemented	verified
+
+1.2.0
+DNSTimeout (dkimpy 1.0)  implemented  verified by inspection
+KeyTable            implemented  verified
+KeytableEd25519     implemented  verified
+MinimumKeyBits      implemented  verified
+SignHeaders	    implemented  verified by inspection
+SigningTable        implemented  verified
+TemporaryDirectory  implemented  verified by inspection
+
+Planned dataset type support (if needed):
 mdb:
 
 Considered for near-term feature release
@@ -49,24 +66,22 @@ Considered for near-term feature release
 AlwaysAddARHeader
 ChangeRootDirectory
 ClockDrift (requires dkimpy change)
-DNSTimeout (requires dkmpy change)
-MacroList
 MilterDebug
-MinimumKeyBits
+OmitHeaders
+OversignHeaders (may require dkimpy changes)
 PeerList
 SignatureAlgorithm
 
 Later
 
 BaseDirectory
+Diagnostics (requires dkimpy changes)
 DontSignMailTo
 ExemptDomains
 ExternalIgnoreList
 FixCRLF
 KeepAuthResults
 KeepTemporaryFiles
-KeyTable
-KeytableEd25519
 LogResults
 LogWhy
 MaximumHeaders
@@ -74,7 +89,6 @@ MaximumSignaturesToVerify
 MultipleSignatures
 MustBeSigned
 NoHeaderB 
-OmitHeaders
 On-BadSignature
 On-Default
 On-DNSError
@@ -82,7 +96,6 @@ On-InternalError
 On-KeyNotFound
 On-NoSignature
 On-SignatureError
-OversignHeaders
 RemoveARAll
 RemoveARFrom
 RemoveOldSignatures 
@@ -90,12 +103,8 @@ RequiredHeaders
 RequireSafeKeys
 SignatureAlgorithm
 SignatureTTL
-SignHeaders
-SigningTable
 SoftwareHeader
 StrictHeaders
-SubDomains
-TemporaryDirectory
 TestDNSData
 TestPublicKeys
 

dkimpy_milter/__init__.py

@@ -1,4 +1,4 @@
-#! /usr/bin/python2
+#! /usr/bin/python3
 # Original dkim-milter.py code:
 # Author: Stuart D. Gathman <stuart@bmsi.com>
 # Copyright 2007 Business Management Systems, Inc.
@@ -25,25 +25,25 @@ import sys
 import syslog
 import Milter
 import dkim
-from dkim.dnsplug import get_txt
-from dkim.util import parse_tag_value
 import authres
 import os
 import tempfile
-import StringIO
+import io
 import re
-from Milter.config import MilterConfigParser
+import codecs
-from Milter.utils import iniplist,parse_addr,parseaddr
+import idna
+from Milter.utils import parse_addr, parseaddr
 import dkimpy_milter.config as config
 from dkimpy_milter.util import drop_privileges
 from dkimpy_milter.util import setExceptHook
 from dkimpy_milter.util import write_pid
-from dkimpy_milter.util import read_keyfile
+from dkimpy_milter.util import get_keys
-from dkimpy_milter.util import own_socketfile
+from dkimpy_milter.util import fold
 
-__version__ = "0.9.3"
+__version__ = "1.2.3"
 FWS = re.compile(r'\r?\n[ \t]+')
 
+
 class dkimMilter(Milter.Base):
     "Milter to check and sign DKIM.  Each connection gets its own instance."
 
@@ -52,92 +52,140 @@ class dkimMilter(Milter.Base):
         self.id = Milter.uniqueID()
         # we don't want config used to change during a connection
         self.conf = milterconfig
-    self.privatersa = privateRSA
-    self.privateed25519 = privateEd25519
         self.fp = None
+        self.fdomain = ''
+        self.iequals = None
 
     @Milter.noreply
-  def connect(self,hostname,unused,hostaddr):
+    def connect(self, hostname, unused, hostaddr):
         self.internal_connection = False
+        self.external_connection = False
         self.hello_name = None
         # sometimes people put extra space in sendmail config, so we strip
-    self.receiver = self.getsymval('j').strip()
+        self.receiver = self.getsymval('j')
+        if self.receiver is not None:
+            self.receiver = self.receiver.strip()
+        try:
+            self.AuthservID = self.conf['AuthservID']
+        except:
+            self.AuthservID = self.receiver
         if hostaddr and len(hostaddr) > 0:
             ipaddr = hostaddr[0]
-        """if iniplist(ipaddr,self.conf.internal_connect): FIXME
+            if self.conf['IntHosts']:
-            self.internal_connection = True"""
+                if self.conf['IntHosts'].match(ipaddr):
-    else: ipaddr = ''
+                    self.internal_connection = True
+        else:
+            ipaddr = ''
         self.connectip = ipaddr
+        if self.conf.get('MacroList') and not self.internal_connection:
+            macrolist = self.conf.get('MacroList')
+            for macro in macrolist:
+                macroname = macro.split('|')[0]
+                macroname = '{' + macroname + '}'
+                macroresult = self.getsymval(macroname)
+                if ((len(macro.split('|')) == 1 and macroresult) or macroresult
+                        in macro.split('|')[1:]):
+                    self.internal_connection = True
+        if self.conf.get('MacroListVerify'):
+            macrolist = self.conf.get('MacroListVerify')
+            for macro in macrolist:
+                macroname = macro.split('|')[0]
+                macroname = '{' + macroname + '}'
+                macroresult = self.getsymval(macroname)
+                if ((len(macro.split('|')) == 1 and macroresult) or macroresult
+                        in macro.split('|')[1:]):
+                    self.external_connection = True
         if self.internal_connection:
             connecttype = 'INTERNAL'
         else:
             connecttype = 'EXTERNAL'
-    if milterconfig.get('Syslog'):
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 1:
-        syslog.syslog("connect from {0} at {1} {2}".format(hostname,hostaddr,connecttype))
+            syslog.syslog("connect from {0} at {1} {2}"
+                          .format(hostname, hostaddr, connecttype))
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3:
+            syslog.syslog("internal_conn: {0}, external_conn: {1}"
+                          .format(self.internal_connection, self.external_connection))
+
         return Milter.CONTINUE
 
     # multiple messages can be received on a single connection
     # envfrom (MAIL FROM in the SMTP protocol) seems to mark the start
     # of each message.
     @Milter.noreply
-  def envfrom(self,f,*str):
+    def envfrom(self, f, *moredata):
-    if milterconfig.get('Syslog'):  
+        try:
-        syslog.syslog("mail from: {0} {1}".format(f,str))
+            f = str(codecs.encode(f, 'UTF-8', 'replace'), 'UTF-8', 'ignore')
-    self.fp = StringIO.StringIO()
+        except TypeError:
+            f = codecs.encode(f, 'UTF-8', 'replace').decode()
+        try:
+            moredata = str(codecs.encode(str(moredata), 'UTF-8', 'replace'), 'UTF-8', 'ignore')
+        except TypeError:
+            moredata = codecs.encode(str(moredata), 'UTF-8', 'replace').decode()
+
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 2:
+            syslog.syslog("mail from: {0} {1}".format(f, moredata))
+        self.fp = io.BytesIO()
         self.mailfrom = f
         t = parse_addr(f)
-    if len(t) == 2: t[1] = t[1].lower()
+        if len(t) == 2:
+            t[1] = t[1].lower()
         self.canon_from = '@'.join(t)
-    self.user = self.getsymval('{auth_authen}')
         self.has_dkim = 0
         self.author = None
         self.arheaders = []
         self.arresults = []
-    '''if self.user:
-      # Very simple SMTP AUTH policy by default:
-      #   any successful authentication is considered INTERNAL
-      self.internal_connection = True
-      auth_type = self.getsymval('{auth_type}')
-      ssl_bits =  self.getsymval('{cipher_bits}')
-      if milterconfig.get('Syslog'):
-          syslog.syslog(
-              "SMTP AUTH:",self.user,"sslbits =",ssl_bits, auth_type,
-              "ssf =",self.getsymval('{auth_ssf}'), "INTERNAL"
-      )
-      # Detailed authorization policy is configured in the access file below.
-      self.arresults.append(
-        authres.SMTPAUTHAuthenticationResult(result = 'pass',
-      	result_comment = auth_type+' sslbits='+ssl_bits, smtp_auth = self.user)
-      )'''
         return Milter.CONTINUE
 
     @Milter.noreply
-  def header(self,name,val):
+    def header(self, name, val):
         lname = name.lower()
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4:
+            if lname == 'content-transfer-encoding':
+                syslog.syslog("content-transfer-encodeing: {0}".format(val))
+            if lname == 'content-type':
+                syslog.syslog("content-type: {0}".format(val))
         if lname == 'dkim-signature':
-        if milterconfig.get('Syslog'):
+            if (self.conf.get('Syslog') and
-          syslog.syslog("{0}: {1}".format(name,val))
+                    self.conf.get('debugLevel') >= 1):
+                syslog.syslog("{0}: {1}".format(name, val))
             self.has_dkim += 1
         if lname == 'from':
-        fname,self.author = parseaddr(val)
+            fname, self.author = parseaddr(idna.alabel(val))
-        self.fdomain = self.author.split('@')[1]
+            try:
-        if milterconfig.get('Syslog'):
+                self.fdomain = self.author.split('@')[1].lower()
-            syslog.syslog("{0}: {1}".format(name,val))
+            except IndexError as er:
+                pass # self.author was not a proper email address
+            # This keeps non-ascii characters out of the From domain
+            try:
+                self.fdomain = str(codecs.encode(self.fdomain, 'ascii', 'replace'), 'ascii', 'ignore')
+            except TypeError:
+                self.fdomain = codecs.encode(self.fdomain, 'ascii', 'replace').decode('ascii','ignore')
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
+                syslog.syslog("{0}: {1}".format(name, val))
         elif lname == 'authentication-results':
             self.arheaders.append(val)
         if self.fp:
-        self.fp.write("%s: %s\n" % (name,val))
+            try:
+                if lname == 'from':
+                    # Non-ascii in email address localpart is legal, so this is a special case
+                    self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'UTF-8', 'replace')))
+                else:
+                    self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'ascii')))
+            except:
+                # Don't choke on header fields with non-ascii garbage in them.
+                pass
         return Milter.CONTINUE
 
     @Milter.noreply
     def eoh(self):
         if self.fp:
-      self.fp.write("\n")                         # terminate headers
+            self.fp.write(b"\n")   # terminate headers
         self.bodysize = 0
         return Milter.CONTINUE
 
     @Milter.noreply
-  def body(self,chunk):         # copy body to temp file
+    def body(self, chunk):        # copy body to temp file
         if self.fp:
             self.fp.write(chunk)  # IOError causes TEMPFAIL in milter
             self.bodysize += len(chunk)
@@ -147,144 +195,331 @@ class dkimMilter(Milter.Base):
         if not self.fp:
             return Milter.ACCEPT  # no message collected - so no eom processing
         # Remove existing Authentication-Results headers for our authserv_id
-    for i,val in enumerate(self.arheaders,1):
+        for i, val in enumerate(self.arheaders, 1):
             # FIXME: don't delete A-R headers from trusted MTAs
             try:
-          ar = authres.AuthenticationResultsHeader.parse_value(FWS.sub('',val))
+                ar = (authres.AuthenticationResultsHeader
-          if ar.authserv_id == self.receiver:
+                      .parse_value(FWS.sub('', val)))
-              self.chgheader('authentication-results',i,'')
+                if ar.authserv_id == self.AuthservID:
-              if milterconfig.get('Syslog'):
+                    self.chgheader('authentication-results', i, '')
+                    if (self.conf.get('Syslog') and
+                            self.conf.get('debugLevel') >= 1):
                         syslog.syslog('REMOVE: {0}'.format(val))
             except:
                 # Don't error out on unparseable AR header fiels
                 pass
-    # Check or sign DKIM
+        # Check and/or sign DKIM
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4):
+            syslog.syslog('self.conf: {0}'.format(self.conf))
         self.fp.seek(0)
-    if (self.fdomain in milterconfig.get('Domain')) and (not milterconfig.get('Mode') == 'v'):
         txt = self.fp.read()
+        self.get_identities_sign()
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3):
+            syslog.syslog('self.domain: {0}, self.fdomain: {1}, self.iequals: {2}'.format(self.domain, self.fdomain, self.iequals))
+        if ((self.fdomain in self.domain) and not self.conf.get('Mode') == 'v'
+                and not self.external_connection):
+            if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3):
+                syslog.syslog("Signing DKIM")
             self.sign_dkim(txt)
-      result = None
+        if ((self.has_dkim) and (not self.internal_connection) and
-    if (self.has_dkim) and (not self.internal_connection) and (milterconfig.get('Mode') == 'v' or milterconfig.get('Mode') == 'sv'):
+            (self.conf.get('Mode') == 'v' or
-      txt = self.fp.read()
+             self.conf.get('Mode') == 'sv')):
             self.check_dkim(txt)
-    else:
-      result = 'none'
         if self.arresults:
-        h = authres.AuthenticationResultsHeader(authserv_id = self.receiver, 
+            h = authres.AuthenticationResultsHeader(authserv_id=
+                                                    self.AuthservID,
                                                     results=self.arresults)
-        h = dkim.fold(str(h))
+            h = fold(codecs.encode(str(h), 'ascii'))
-        if milterconfig.get('Syslog'):
+            if (self.conf.get('Syslog') and
-            syslog.syslog(str(h))
+                    self.conf.get('debugLevel') >= 2):
-        name,val = str(h).split(': ',1)
+                syslog.syslog(codecs.decode(h, 'ascii'))
-        self.addheader(name,val,0)
+            name, val = codecs.decode(h, 'ascii').split(': ', 1)
+            self.addheader(name, val, 0)
         return Milter.CONTINUE
 
-  def sign_dkim(self,txt):
+    # get parent domain to be signed for if fdomain is a subdomain
-      canon = milterconfig.get('Canonicalization')
+    def get_parent_domain(self, fdomain, domains):
+        for domain in domains:
+            rhs = '.'+domain
+            # compare right hand side of fdomain against .domain
+            if fdomain[-len(rhs):] == rhs:
+                # return parent domain on match
+                syslog.syslog('domain: {0}'.format(domain))
+                return domain
+        # or return the fdomain itself
+        return fdomain
+
+    def get_identities_sign(self):
+        """Determine d= and i= identiies for signature"""
+        self.domain = []
+        self.iequals = None
+        try:
+            self.privkeyRSA = self.conf.get('privateRSA')
+        except:
+            self.privkeyRSA = ''
+        try:
+            self.privkeyEd25519 = self.conf.get('privateEd25519')
+        except:
+            self.privkeyEd25519 = ''
+        try:
+            self.selectorRSA = self.conf.get('Selector')
+        except:
+            self.selectorRSA = ''
+        try:
+            self.selectorEd25519 = self.conf.get('SelectorEd25519')
+        except:
+            self.selectorEd25519 = ''
+        if not self.domain and self.conf.get('Domain'):
+            self.domain = self.conf.get('Domain')
+        if self.conf.get('SubDomains'):
+            self.fdomain = self.get_parent_domain(self.fdomain, self.domain)
+        if self.conf.get('SigningTable'):
+            match = False
+            for dictkey, dictvalues in self.conf.get('SigningTable').items():
+                if dictkey == '%':
+                    self.domain.append(self.fdomain)
+                    match = True
+                elif len(dictkey.split('*')) == 1:
+                    if dictkey == self.author:
+                        self.domain.append(self.fdomain)
+                        match = True
+                else:
+                    if len(dictkey.split('*')) == 2:
+                        if dictkey.split('*')[1] == self.author[-len(dictkey.split('*')[1]):]:
+                            self.domain.append(self.fdomain)
+                            match = True
+                    self.domain.append(self.fdomain)
+                try:
+                    if len(dictvalues) == 2 and match:
+                        if dictvalues[0] =='%':
+                            self.iequals = codecs.encode('@' + self.fdomain)
+                        elif dictvalues[0][1:] == self.fdomain or self.get_parent_domain(dictvalues[0][1:], self.domain) == self.fdomain:
+                            self.iequals = codecs.encode(dictvalues[0])
+                except IndexError:
+                    pass
+                if match:
+                    #TODO add KeyTable stuffs here.
+                    keytablekey = dictvalues[-1] # Last value in the SigningTable row.
+                    if self.conf.get('privateRSATable'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateRSATable')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.domain.append(self.fdomain)
+                            self.selectorRSA = keytabledata[1]
+                            self.privkeyRSA = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    if self.conf.get('privateEd25519Table'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateEd25519Table')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.domain.append(self.fdomain)
+                            self.selectorEd25519 = keytabledata[1]
+                            self.privkeyEd25519 = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    if (self.fdomain == '%'):
+                        self.fdomain = self.author.split('@')[1].lower()
+                    break
+
+    def sign_dkim(self, txt):
+        canon = codecs.encode(self.conf.get('Canonicalization'), 'ascii')
         canonicalize = []
-      if len(canon.split('/')) == 2:
+        if len(canon.split(b'/')) == 2:
-          canonicalize.append(canon.split('/')[0])
+            canonicalize.append(canon.split(b'/')[0])
-          canonicalize.append(canon.split('/')[1])
+            canonicalize.append(canon.split(b'/')[1])
         else:
             canonicalize.append(canon)
             canonicalize.append(canon)
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog('canonicalize: {0}'.format(canonicalize))
+        sign_headers = self.conf.get('SignHeaders')
+        if not sign_headers:
+            # None or empty. DKIM explicitly tests for None.
+            sign_headers = None
         try:
+            if self.privkeyRSA:
                 d = dkim.DKIM(txt)
-        h = d.sign(milterconfig.get('Selector'), self.fdomain, privateRSA,
+                h = d.sign(codecs.encode(self.selectorRSA, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
-                canonicalize=(canonicalize[0], canonicalize[1]))
+                           codecs.encode(self.privkeyRSA, 'ascii'),
-        name,val = h.split(': ',1)
+                           canonicalize=(canonicalize[0], canonicalize[1]),
-        self.addheader(name,val.strip().replace('\r\n','\n'),0)
+                           identity=self.iequals, include_headers=sign_headers)
-        if privateEd25519:
+                name, val = h.split(b': ', 1)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
+                if (self.conf.get('Syslog') and
+                    (self.conf.get('SyslogSuccess')
+                     or self.conf.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
+            if self.privkeyEd25519:
                 d = dkim.DKIM(txt)
-            h = d.sign(milterconfig.get('SelectorEd25519'), self.fdomain, privateEd25519,
+                h = d.sign(codecs.encode(self.selectorEd25519, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
-                    canonicalize=(canonicalize[0], canonicalize[1]), signature_algorithm='ed25519-sha256')
+                           self.privkeyEd25519,
-            name,val = h.split(': ',1)
+                           canonicalize=(canonicalize[0], canonicalize[1]),
-            self.addheader(name,val.strip().replace('\r\n','\n'),0)
+                           identity=self.iequals, include_headers=sign_headers,
+                           signature_algorithm=b'ed25519-sha256')
+                name, val = h.split(b': ', 1)
+                self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
+                if (self.conf.get('Syslog') and
+                    (self.conf.get('SyslogSuccess')
+                     or self.conf.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature added (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
         except dkim.DKIMException as x:
-          if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog('DKIM: {0}'.format(x))
         except Exception as x:
-          if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog("sign_dkim: {0}".format(x))
             raise
 
-  def check_dkim(self,txt):
+    def check_dkim(self, txt):
         res = False
-      conf = self.conf
+        self.header_a = None
         for y in range(self.has_dkim):  # Verify _ALL_ the signatures
-          d = dkim.DKIM(txt)
+            d = dkim.DKIM(txt, minkey=self.conf.get('MinimumKeyBits'), timeout=self.conf.get('DNSTimeout'))
             try:
-            res = d.verify(idx=y)
+                dnsoverride = self.conf.get('DNSOverride')
-            if res:
+                if isinstance(dnsoverride, str):
-              self.dkim_comment = 'Good {0} bit {1} signature.'.format(d.keysize, d.signature_fields.get(b'a'))
+                    timeout = 5
+                    domain = self.fdomain
+                    def dnsfunc(domain, timeout=timeout, dnsoverride=dnsoverride):
+                        return dnsoverride
+                    syslog.syslog("DNSOverride: {0}".format(dnsoverride))
+                    res = d.verify(idx=y, dnsfunc=dnsfunc)
                 else:
-              self.dkim_comment = 'Bad {0} bit {1} signature.'.format(d.keysize, d.signature_fields.get(b'a'))
+                    res = d.verify(idx=y)
+                algo = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
+                if res:
+                    if algo == 'ed25519-sha256':
+                        self.dkim_comment = ('Good {0} signature'
+                                             .format(algo))
+                    else:
+                        self.dkim_comment = ('Good {0} bit {1} signature'
+                                             .format(d.keysize, algo))
+                else:
+                    self.dkim_comment = ('Bad {0} bit {1} signature.'
+                                         .format(d.keysize, algo))
             except dkim.DKIMException as x:
                 self.dkim_comment = str(x)
-            if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
                     syslog.syslog('DKIM: {0}'.format(x))
             except Exception as x:
                 self.dkim_comment = str(x)
-            if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
-                syslog.syslog("check_dkim: {0}".format(x))
+                    syslog.syslog("check_dkim: Internal program fault while verifying: {0}".format(x))
-          self.header_i = d.signature_fields.get(b'i')
+            try:
-          self.header_d = d.signature_fields.get(b'd')
+                # i= is optional and dkimpy is fine if it's not provided
-          self.header_a = d.signature_fields.get(b'a')
+                self.header_i = codecs.decode(d.signature_fields.get(b'i'), 'ascii')
+            except TypeError as x:
+                self.header_i = None
+            try:
+                self.header_d = codecs.decode(d.signature_fields.get(b'd'), 'ascii')
+                self.header_a = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
+            except Exception as x:
+                self.dkim_comment = str(x)
+                if self.conf.get('Syslog'):
+                    syslog.syslog("check_dkim: Internal program fault extracting header a or d: {0}".format(x))
+                self.header_d = None
+            if not self.header_a:
+                self.header_a = 'rsa-sha256'
             if res:
-              if milterconfig.get('Syslog'):
+                if (self.conf.get('Syslog') and
-                  syslog.syslog('DKIM: Pass ({0})'.format(d.domain))
+                        (self.conf.get('SyslogSuccess') or
-              self.dkim_domain = d.domain
+                         self.conf.get('debugLevel') >= 1)):
+                    syslog.syslog('{0}: {1} DKIM signature verified (s={2} '
+                                  'd={3})'.format(self.getsymval('i'),
+                                                  d.signature_fields.get(b'a').decode(),
+                                                  d.signature_fields.get(b's').decode(),
+                                                  d.domain.decode().lower()))
+                self.dkim_domain = d.domain.lower()
             else:
-            fd,fname = tempfile.mkstemp(".dkim")
+                if self.conf.get('DiagnosticDirectory'):
-            with os.fdopen(fd,"w+b") as fp:
+                    tempfile.tempdir = self.conf.get('DiagnosticDirectory')
+                    fd, fname = tempfile.mkstemp(".dkim")
+                    with os.fdopen(fd, "w+b") as fp:
                         fp.write(txt)
-            if milterconfig.get('Syslog'):
+                    if self.conf.get('Syslog'):
-                syslog.syslog('DKIM: Fail (saved as {0})'.format(fname))
+                        syslog.syslog('DKIM: Fail (saved as {0})'
+                                      .format(fname))
+                else:
+                    if self.conf.get('Syslog'):
+                        if d.domain:
+                            syslog.syslog('DKIM: Fail ({0})'
+                                          .format(d.domain.lower()))
+                        else:
+                            syslog.syslog('DKIM: Fail, unextractable domain')
             if res:
                 result = 'pass'
             else:
                 result = 'fail'
+            res = False
+            if self.header_d:
                 self.arresults.append(
                     authres.DKIMAuthenticationResult(result=result,
-              header_i = self.header_i, header_d = self.header_d, header_a = self.header_a,
+                                                 header_i=self.header_i,
-              result_comment = self.dkim_comment)
+                                                 header_d=self.header_d,
+                                                 header_a=self.header_a,
+                                                 result_comment=
+                                                 self.dkim_comment)
             )
+            self.header_a = None
         return
 
 def main():
     # Ugh, but there's no easy way around this.
     global milterconfig
-    global privateRSA
+    configFile = '/usr/local/etc/dkimpy-milter.conf'
-    global privateEd25519
-    privateRSA = False
-    privateEd25519 = False
-    configFile = '/etc/dkimpy-milter.conf'
     if len(sys.argv) > 1:
-        if sys.argv[1] in ( '-?', '--help', '-h' ):
+        if (sys.argv[1] in ('-?', '--help', '-h')) or len(sys.argv) == 3 or \
-            print('usage: dkimpy-milter [<configfilename>]')
+               (len(sys.argv) == 4 and sys.argv[2] != '-P'):
+            print('usage: dkimpy-milter [<configfilename> [-P <pidfile>]]')
             sys.exit(1)
         configFile = sys.argv[1]
-    milterconfig = config._processConfigFile(filename = configFile)
+    milterconfig = config._processConfigFile(filename=configFile)
+    if len(sys.argv) == 4:
+        if sys.argv[2] == '-P':
+            # Command line PID file argument overrides config file
+            milterconfig['PidFile'] = sys.argv[3]
     if milterconfig.get('Syslog'):
-        facility = eval("syslog.LOG_{0}".format(milterconfig.get('SyslogFacility').upper()))
+        facility = eval("syslog.LOG_{0}"
+                        .format(milterconfig.get('SyslogFacility').upper()))
         syslog.openlog(os.path.basename(sys.argv[0]), syslog.LOG_PID, facility)
         setExceptHook()
     pid = write_pid(milterconfig)
-    if milterconfig.get('KeyFile'):
+    milterconfig = get_keys(milterconfig)
-        privateRSA = read_keyfile(milterconfig, 'RSA')
-    if milterconfig.get('KeyFileEd25519'):
-        privateEd25519 = read_keyfile(milterconfig, 'Ed25519')
     Milter.factory = dkimMilter
     Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
     miltername = 'dkimpy-filter'
     socketname = milterconfig.get('Socket')
-    if milterconfig.get('Syslog'):
+    if socketname is None:
-        syslog.syslog('dkimpy-milter started:{0} user:{1}'.format(pid,milterconfig.get('UserID')))
+        if int(os.environ.get('LISTEN_PID', '0')) == os.getpid():
+            lfds = os.environ.get('LISTEN_FDS')
+            if lfds is not None:
+                if lfds != '1':
+                    syslog.syslog('LISTEN_FDS is set to "{0}", but we only know how to deal with "1", ignoring it'.
+                                  format(lfds))
+                else:
+                    socketname = 'fd:3'
+        if socketname is None:
+            socketname = 'local:/var/run/dkimpy-milter/dkimpy-milter.sock'
     sys.stdout.flush()
-    Milter.runmilter(miltername,socketname,240)
+    if milterconfig.get('Syslog'):
-    own_socketfile(milterconfig)
+        syslog.syslog('dkimpy-milter starting:{0} user:{1}'
+                      .format(pid, milterconfig.get('UserID')))
     drop_privileges(milterconfig)
+    Milter.runmilter(miltername, socketname, 240)
 
 if __name__ == "__main__":
     main()

dkimpy_milter/__main__.py

@@ -0,0 +1,6 @@
+#!/usr/bin/python3
+
+from dkimpy_milter import main
+
+if __name__ == "__main__":
+    main()

dkimpy_milter/config.py

@@ -27,42 +27,231 @@
 import syslog
 import os
 import sys
-import re
-import urllib
 import stat
 import dkim
-
+import socket
+import ipaddress
+from .dnsplug import Session
 
 #  default values
 defaultConfigData = {
-        'Syslog' : 'yes',
+    'Syslog': 'yes',
-        'SyslogFacility' : 'mail',
+    'SyslogFacility': 'mail',
-        'UMask' : 007,
+    'UMask': 0o07,
-        'Mode' : 'sv',
+    'Mode': 'sv',
-        'Socket' : 'local:/var/run/dkimpy-milter/dkimpy-milter.sock',
+    'MinimumKeyBits': 1024,
-        'PidFile'  : '/var/run/dkimpy-milter/dkimpy-milter.pid',
+    'Socket': None,
-        'UserID' : 'dkimpy-milter',
+    'PidFile': None,
-        'Canonicalization' : 'relaxed/simple'
+    'UserID': 'dkimpy-milter',
+    'Canonicalization': 'relaxed/simple',
+    'InternalHosts': '127.0.0.1',
+    'IntHosts': False,
+    'DiagnosticDirectory': '',
+    'MacroList': '',
+    'MacroListVerify': '',
+    'DNSOverride': None,
+    'DNSTimeout': 5,
+    'SubDomains': False,
+    'SigningTable': None,
+    'debugLevel': 0  # Undocumented config item for developer use
     }
 
 
-#################################
 class ConfigException(Exception):
     '''Exception raised when there's a configuration file error.'''
     pass
 
-####################################################################
+
-def _processConfigFile(filename = None, configdata = None, useSyslog = 1,
+class HostsDataset(object):
-        useStderr = 0):
+    '''Hold a group of host related dataset objects'''
+
+    def __init__(self, dataset):
+        self.dataset = []
+        # Self.dataset will end up being a list of DataSetItem(s).
+        for item in dataset:
+            item = item.rstrip(']')
+            item = item.lstrip('[')
+            self.dataset.append(self.DatasetItem(item))
+
+    class DatasetItem(object):
+        '''Individual dataset item'''
+
+        def __init__(self, item):
+            self.item = item
+            self.isipv4 = False
+            self.isipv4cidr = False
+            self.isipv6 = False
+            self.isipv6cidr = False
+            self.ishostname = False
+            self.isdomain = False
+            self.negative = False
+            if self.item[0] == '!':
+                self.item = item[1:]
+                self.negative = True
+            try:
+                try:
+                    self.item = ipaddress.ip_address(str(self.item, "utf-8"))
+                except TypeError:
+                    self.item = ipaddress.ip_address(self.item)
+                if isinstance(self.item, ipaddress.IPv4Address):
+                    self.isipv4 = True
+                elif isinstance(self.item, ipaddress.IPv6Address):
+                    self.isipv6 = True
+            except ValueError as e:
+                try:
+                    try:
+                        self.item = ipaddress.ip_network(str
+                                                         (self.item, "utf-8"),
+                                                         strict=False)
+                    except TypeError:
+                        self.item = ipaddress.ip_network(self.item,
+                                                         strict=False)
+                    if isinstance(self.item, ipaddress.IPv4Network):
+                        self.isipv4cidr = True
+                    elif isinstance(self.item, ipaddress.IPv6Network):
+                        self.isipv6cidr = True
+                except ValueError as e2:
+                    if self.item[0] == '.' and len(self.item.split('.')) > 2:
+                        self.isdomain = True
+                    elif len(self.item.split('.')) > 1:  # It has a '.' in it
+                        self.ishostname = True
+                    else:
+                        raise ConfigException('Unknown dataset item: {0}'
+                                              .format(item))
+
+    def match(self, connectip):
+        '''Check if the connect IP is part of the dataset'''
+        try:
+            source = ipaddress.ip_address(str(connectip, "utf-8"))
+        except TypeError:
+            source = ipaddress.ip_address(connectip)
+        for item in self.dataset:
+            if item.isdomain or item.ishostname:
+                result = self.matchname(source)   # Match host/domains first
+                if result:
+                    return(result)
+            elif item.isipv4 or item.isipv4cidr:  # Then IPv4/6 addresses or
+                if isinstance(source, ipaddress.IPv4Address):  # networks
+                    return(self.match4(source))   # depending on the item type
+            elif item.isipv6 or item.isipv6cidr:  # and connect type
+                if isinstance(source, ipaddress.IPv6Address):
+                    return(self.match6(source))
+
+    def matchname(self, source):
+        '''Does source IP address relate to a domain/hostname in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchdomain = False
+        negativedomain = False
+        ptrlist = self.getptr(source)
+        for item in self.dataset:
+            if item.isdomain:
+                for ptr in ptrlist:
+                    # Strip the leading '.' off the domain name for exact match
+                    if item.item[1:] == ptr[-len(item.item)+1:]:
+                        matchdomain = True
+                        negativedomain = item.negative
+            elif item.ishostname:
+                for ptr in ptrlist:
+                    if item.item == ptr:
+                        matchone = True
+                        negativeone = item.negative
+        if matchdomain and not negativedomain:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+    def getptr(self, source):
+        '''Get validated PTR name of IP address'''
+        results = []
+        s = Session()
+        ptrnames = s.dns(source.reverse_pointer, 'PTR', timeout=self.conf.get('DNSTimeout'))
+        for name in ptrnames:
+            if isinstance(source, ipaddress.IPv4Address):
+                ips = s.dns(name, 'A')
+                for ip in ips:
+                    try:
+                        ip = ipaddress.IPv4Address(str(ip, 'UTF-8'))
+                    except TypeError:
+                        ip = ipaddress.IPv4Address(ip)
+                    if ip == source:
+                        results.append(name)
+            if isinstance(source, ipaddress.IPv6Address):
+                ips = s.dns(name, 'AAAA')
+                for ip in ips:
+                    try:
+                        ip = ipaddress.IPv6Address(str(ip, 'UTF-8'))
+                    except TypeError:
+                        ip = ipaddress.IPv6Address(ip)
+                if ip == source:
+                    results.append(name)
+        return results
+
+    def match4(self, source):
+        '''Is the source IP related to a IPv4 address/network in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchcidr = False
+        negativecidr = False
+        for item in self.dataset:
+            if item.isipv4:
+                if source == item.item:
+                    matchone = True
+                    negativeone = item.negative
+            elif item.isipv4cidr:
+                if source in item.item:
+                    matchcidr = True
+                    negativecidr = item.negative
+        if matchcidr and not negativecidr:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+    def match6(self, source):
+        '''Is the source IP realted to a IPv6 address/network in the dataset'''
+        match = False
+        matchone = False
+        negativeone = False
+        matchcidr = False
+        negativecidr = False
+        for item in self.dataset:
+            if item.isipv6:
+                if source == item.item:
+                    matchone = True
+                    negativeone = item.negative
+            elif item.isipv6cidr:
+                if source in item.item:
+                    matchcidr = True
+                    negativecidr = item.negative
+        if matchcidr and not negativecidr:
+            match = True
+        if matchone and not negativeone:
+            return True
+        if matchone and negativeone:
+            match = False
+        return(match)
+
+
+def _processConfigFile(filename=None, configdata=None, useSyslog=1,
+                       useStderr=0):
     '''Load the specified config file, exit and log errors if it fails,
     otherwise return a config dictionary.'''
 
-    import config
+    from . import config
-    if configdata == None: configdata = config.defaultConfigData
+    if configdata is None:
-    if filename != None:
+        configdata = config.defaultConfigData
+    if filename is not None:
         try:
             _readConfigFile(filename, configdata)
-        except Exception, e:
+        except Exception as e:
             raise
             if useSyslog:
                 syslog.syslog(e.args[0])
@@ -71,7 +260,7 @@ def _processConfigFile(filename = None, configdata = None, useSyslog = 1,
             sys.exit(1)
     return(configdata)
 
-####################
+
 def _find_boolean(item):
     if type(item) == int:
         item = str(item)
@@ -84,9 +273,16 @@ def _find_boolean(item):
     return item
 
 
+def _make_authserv_id(as_id):
+    """Determine AuthservID if needed"""
+    if as_id == 'HOSTNAME':
+        as_id = socket.gethostname()
+    return as_id
+
 def _dataset_to_list(dataset):
     """Convert a dataset (as defined in dkimpymilter.8) and return a python
-       list of values."""
+       list of values.  For multiline datasets like KeyTable and SigningTable a
+       key : values dictionary is returned"""
     if not isinstance(dataset, str):
         # If it was a csl with more than one value, it's already a list, we
         # only need to remove the name from the first value.
@@ -96,85 +292,128 @@ def _dataset_to_list(dataset):
             dataset[dataset.index(item)] = item.strip().strip(',')
         return dataset
     elif isinstance(dataset, str):
-        if dataset[0] == '/' or dataset[:5] == 'file:':
+        if dataset[0] == '/' or dataset[:5] == 'file:' or dataset[:7] == 'refile:':
-            # This is a flat file dataset
+            # This is a flat file dataset, which are key value:value stores
             ds = []
-            if dataset[0] == '/':
+            dsd = {}
+            if dataset[0] == '/' or dataset[:2] == './' or dataset[:3] == '../':
                 dsname = dataset
-            if dataset[:5] == 'file:':
+            elif dataset[:5] == 'file:':
                 dsname = dataset[5:]
+            elif dataset[:7] == 'refile:':
+                dsname = dataset[7:]
             dsf = open(dsname, 'r')
             for line in dsf.readlines():
                 if line[0] != '#':
+                    if len(line.split()) == 1:
                         if len(line.split(':')) == 1:
                             ds.append(line.strip())
                         else:
                             for element in line.split(':'):
                                 ds.append(element.strip().strip(':'))
+                    elif len(line.split()) == 2: # key value:value:value
+                        key, values = line.split()
+                        values = values.split(':')
+                        dsd.update({key:values})
             dsf.close()
+            if ds:
                 return ds
+            elif dsd:
+                return dsd
         # If it's a str and csl, it has one value and we return a list
         if dataset[:4] == 'csl:':
-            return [dataset[4:].strip().strip(',')]
+            datalist = dataset[4:].split(',')
+            for item in datalist:
+                datalist[datalist.index(item)] = item.strip().strip(',')
+            return datalist
         else:
-            return [dataset.strip().strip(',')]
+            datalist = dataset.split(',')
+            for item in datalist:
+                datalist[datalist.index(item)] = item.strip().strip(',')
+            return datalist
+        if dataset[-3:] == '.db' or dataset[:3] == 'db:':
+            #  This is a Sleepycat (Oracle) DB  dataset, which we dont support
+            raise dkim.ParameterError('Unsupported dataset db datase: {0}'
+                                      .format(type(dataset)))
 
-    raise dkim.ParameterError('Unimplmented dataset type: {0}'.format(type(dataset)))
+    raise dkim.ParameterError('Unimplmented dataset type: {0}'
+                              .format(type(dataset)))
 
-###############################################################
+
-commentRx = re.compile(r'^(.*)#.*$')
+def _readConfigFile(path, configData=None, configGlobal={}):
-def _readConfigFile(path, configData = None, configGlobal = {}):
     '''Reads a configuration file from the specified path, merging it
     with the configuration data specified in configData.  Returns a
     dictionary of name/value pairs based on configData and the values
     read from path.'''
 
-    debugLevel = configGlobal.get('debugLevel', 0)
+    # No config file data is available yet, so to debug _readConfigFile, set
-    if debugLevel >= 5: syslog.syslog('readConfigFile: Loading "%s"' % path)
+    # the value here.
-    if configData == None: configData = {}
+    debugLevel = 0
+    if debugLevel >= 5:
+        syslog.syslog('readConfigFile: Loading "%s"' % path)
+    if configData is None:
+        configData = {}
     nameConversion = {
-        'AuthservID' : 'str',
+        'AuthservID': 'str',
-        'Syslog' : 'bool',
+        'Syslog': 'bool',
-        'SyslogFacility' : 'str',
+        'SyslogFacility': 'str',
-        'SyslogSuccess' : 'bool',
+        'SyslogSuccess': 'bool',
-        'UMask' : 'int',
+        'UMask': 'int',
-        'Mode' : 'str',
+        'Mode': 'str',
-        'Socket' : 'str',
+        'MinimumKeyBits': 'int',
-        'PidFile'  : 'str',
+        'Socket': 'str',
-        'UserID' : 'str',
+        'PidFile': 'str',
-        'Domain' : 'dataset',
+        'UserID': 'str',
-        'KeyFile' : 'str',
+        'Domain': 'dataset',
-        'KeyFileEd25519' : 'str',
+        'SubDomains': 'bool',
-        'Selector' : 'str',
+        'KeyFile': 'str',
+        'KeyTable': 'dataset',
+        'KeyFileEd25519': 'str',
+        'KeyTableEd25519': 'dataset',
+        'Selector': 'str',
         'SelectorEd25519': 'str',
-        'Canonicalization' : 'str',
+        'SigningTable': 'dataset',
+        'Canonicalization': 'str',
+        'InternalHosts': 'dataset',
+        'IntHosts': 'bool',
+        'DiagnosticDirectory': 'str',
+        'MacroList': 'dataset',
+        'MacroListVerify': 'dataset',
+        'DNSOverride': 'str',
+        'DNSTimeout': 'int',
+        'debugLevel': 'int',
+        'SignHeaders': 'dataset'
         }
 
     #  check to see if it's a file
     try:
         mode = os.stat(path)[0]
-    except OSError, e:
+    except OSError as e:
-        syslog.syslog(syslog.LOG_ERR,'ERROR stating "%s": %s' % ( path, e.strerror ))
+        syslog.syslog(syslog.LOG_ERR, 'ERROR stating "%s": %s'
+                      % (path, e.strerror))
         return(configData)
     if not stat.S_ISREG(mode):
-        syslog.syslog(syslog.LOG_ERR,'ERROR: is not a file: "%s", mode=%s' % ( path, oct(mode) ))
+        syslog.syslog(syslog.LOG_ERR, 'ERROR: is not a file: "%s", mode=%s'
+                      % (path, oct(mode)))
         return(configData)
 
     #  load file
     fp = open(path, 'r')
     while 1:
         line = fp.readline()
-        if not line: break
+        if not line:
+            break
 
         #  parse line
         line = line.split('#', 1)[0].strip()
-        if not line: continue
+        if not line:
+            continue
         data = line.split()
         if len(data) != 2:
             if len(data) == 1:
                 if debugLevel >= 1:
-                    syslog.syslog('Configuration item "%s" not defined in file "%s"'
+                    syslog.syslog('Config item "%s" not defined in file "%s"'
-                        % ( line, path ))
+                                  % (line, path))
         if len(data) == 1:
             name = data
             value = ''
@@ -185,24 +424,51 @@ def _readConfigFile(path, configData = None, configGlobal = {}):
             value = data[1:]
 
         #  check validity of name
+        try:
             conversion = nameConversion.get(name)
-        if conversion == None:
+        except TypeError:
-            syslog.syslog('ERROR: Unknown name "%s" in file "%s"' % ( name, path ))
+            name = name[0]
+            syslog.syslog('Config item "%s" does not provide a value in file "%s"'
+                          % (name, path))
+            conversion = None
+        if conversion is None:
+            syslog.syslog('ERROR: Unknown name or name missing value "%s" in file "%s"'
+                          % (name, path))
             continue
 
-        if debugLevel >= 5: syslog.syslog('readConfigFile: Found entry "%s=%s"'
+        if debugLevel >= 5:
-                % ( name, value ))
+            syslog.syslog('readConfigFile: Found entry "%s=%s"'
+                          % (name, value))
         if conversion == 'bool':
             configData[name] = _find_boolean(value)
         elif conversion == 'str':
+            if isinstance(value, list):
+                configData[name] = line.split(None, 1)[1]
+            else:
                 configData[name] = str(value)
         elif conversion == 'int':
+            if name == 'MinimumKeyBits':
+                if int(value) == 0:
+                    # Odd inheritence from OpenDKIM where value of 0 means use default.
+                    value = configData.get(name)
             configData[name] = int(value)
         elif conversion == 'dataset':
             configData[name] = _dataset_to_list(value)
         else:
-            syslog.syslog(str('name: ' + name + ' value: ' + value + ' conversion: ' + conversion))
+            syslog.syslog(str('name: ' + name + ' value: ' + value +
+                              ' conversion: ' + conversion))
             configData[name] = conversion(value)
     fp.close()
+    try:
+        configData['AuthservID'] = _make_authserv_id(configData.get('AuthservID', 'HOSTNAME'))
+    except Exception as e:
+        syslog.syslog("Could not make AuthservID: {}".format(e))
+        pass
+
+    try:
+        configData['IntHosts'] = HostsDataset(configData['InternalHosts'])
+    except Exception as e:
+        syslog.syslog("Could not make HostDataset from InternalHosts: {}".format(e))
+        pass
 
     return(configData)

dkimpy_milter/dnsplug.py

@@ -0,0 +1,168 @@
+## @package dnsplug
+# Provide a higher level interface to pydns or dnspython (or other provider).
+# NOT RELEASED: this is a proposed API and implementation.
+# Goals - work with both pydns and dnspython (and possibly other libraries)
+# at a simplied level.
+# TODO:
+# 1. map exceptions to common dnsplug.DNSError exception (with
+#    original exception saved as a member).
+# 2. include dict based implementation (handy for test suites)
+# 3. move implementations to subpackages to enable autoselect on first call.
+
+## Maximum number of CNAME records to follow
+MAX_CNAME = 10
+
+## Lookup DNS records by label and RR type.
+# The response can include records of other types that the DNS
+# server thinks we might need.  FIXME: empty result
+# could mean NXDOMAIN or NOANSWER.
+# @param name the DNS label to lookup
+# @param qtype the name of the DNS RR type to lookup
+# @param tcpfallback if False, raise exception instead of TCP fallback
+# @return a list of ((name,type),data) tuples
+def DNSLookup(name, qtype, tcpfallback=True, timeout=30):
+    raise NotImplementedError('No supported dns library found')
+
+class Session(object):
+  """A Session object has a simple cache with no TTL that is valid
+   for a single "session", for example an SMTP conversation."""
+  def __init__(self):
+    self.cache = {}
+
+  ## Additional DNS RRs we can safely cache.
+  # We have to be careful which additional DNS RRs we cache.  For
+  # instance, PTR records are controlled by the connecting IP, and they
+  # could poison our local cache with bogus A and MX records.  
+  # Each entry is a tuple of (query_type,rr_type).  So for instance,
+  # the entry ('MX','A') says it is safe (for milter purposes) to cache
+  # any 'A' RRs found in an 'MX' query.
+  SAFE2CACHE = frozenset((
+    ('MX','MX'), ('MX','A'),
+    ('CNAME','CNAME'), ('CNAME','A'),
+    ('A','A'),
+    ('AAAA','AAAA'),
+    ('PTR','PTR'),
+    ('NS','NS'), ('NS','A'),
+    ('TXT','TXT'),
+    ('SPF','SPF')
+  ))
+
+  ## Cached DNS lookup.
+  # @param name the DNS label to query
+  # @param qtype the query type, e.g. 'A'
+  # @param cnames tracks CNAMES already followed in recursive calls
+  def dns(self, name, qtype, cnames=None):
+    """DNS query.
+
+    If the result is in cache, return that.  Otherwise pull the
+    result from DNS, and cache ALL answers, so additional info
+    is available for further queries later.
+
+    CNAMEs are followed.
+
+    If there is no data, [] is returned.
+
+    pre: qtype in ['A', 'AAAA', 'MX', 'PTR', 'TXT', 'SPF']
+    post: isinstance(__return__, types.ListType)
+    """
+    result = self.cache.get( (name, qtype) )
+    cname = None
+
+    if not result:
+        safe2cache = Session.SAFE2CACHE
+        for k, v in DNSLookup(name, qtype):
+            if k == (name, 'CNAME'):
+                cname = v
+            if (qtype,k[1]) in safe2cache:
+                self.cache.setdefault(k, []).append(v)
+        result = self.cache.get( (name, qtype), [])
+    if not result and cname:
+        if not cnames:
+            cnames = {}
+        elif len(cnames) >= MAX_CNAME:
+            #return result    # if too many == NX_DOMAIN
+            raise DNSError('Length of CNAME chain exceeds %d' % MAX_CNAME)
+        cnames[name] = cname
+        if cname in cnames:
+            raise DNSError('CNAME loop')
+        result = self.dns(cname, qtype, cnames=cnames)
+    return result
+
+def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=5):
+    try:
+	# FIXME: To be thread safe, we create a fresh DnsRequest with
+	# each call.  It would be more efficient to reuse
+	# a req object stored in a Session.
+        req = DNS.DnsRequest(name, qtype=qtype, timeout=timeout)
+        resp = req.req()
+        #resp.show()
+        # key k: ('wayforward.net', 'A'), value v
+        # FIXME: pydns returns AAAA RR as 16 byte binary string, but
+        # A RR as dotted quad.  For consistency, this driver should
+        # return both as binary string.
+        #
+        if resp.header['tc'] == True:
+          if not tcpfallback:
+              raise DNS.DNSError('DNS: Truncated UDP Reply, SPF records should fit in a UDP packet')
+          try:
+              req = DNS.DnsRequest(name, qtype=qtype, protocol='tcp',
+                        timeout=timeout)
+              resp = req.req()
+          except DNS.DNSError as x:
+              raise DNS.DNSError('TCP Fallback error: ' + str(x))
+        return [((a['name'], a['typename']), a['data']) for a in resp.answers]
+    except IOError as x:
+        raise DNS.DNSError('DNS: ' + str(x))
+
+def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=5):
+  retVal = []
+  try:
+    # FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
+    answers = dns.resolver.query(name, qtype, raise_on_no_answer=False, lifetime=timeout)
+    for rdata in answers:
+      if qtype == 'A' or qtype == 'AAAA':
+        retVal.append(((name, qtype), rdata.address))
+      elif qtype == 'MX':
+        retVal.append(((name, qtype), (rdata.preference, rdata.exchange)))
+      elif qtype == 'PTR':
+        retVal.append(((name, qtype), rdata.target.to_text(True)))
+      elif qtype == 'TXT' or qtype == 'SPF':
+        retVal.append(((name, qtype), list(rdata.strings)))
+  except dns.resolver.NoAnswer:
+    pass
+  except dns.resolver.NXDOMAIN:
+    pass
+  return retVal
+
+try:
+    # prefer dnspython (the more complete library)
+    import dns
+    import dns.resolver  # http://www.dnspython.org
+    import dns.exception
+
+    if not hasattr(dns.rdatatype,'SPF'):
+      # patch in type99 support
+      dns.rdatatype.SPF = 99
+      dns.rdatatype._by_text['SPF'] = dns.rdatatype.SPF
+
+    DNSLookup = DNSLookup_dnspython
+except:
+    import DNS    # http://pydns.sourceforge.net
+
+    if not hasattr(DNS.Type, 'SPF'):
+        # patch in type99 support
+        DNS.Type.SPF = 99
+        DNS.Type.typemap[99] = 'SPF'
+        DNS.Lib.RRunpacker.getSPFdata = DNS.Lib.RRunpacker.getTXTdata
+
+    # Fails on Mac OS X? Add domain to /etc/resolv.conf
+    DNS.DiscoverNameServers()
+
+    DNSLookup = DNSLookup_pydns
+
+if __name__ == '__main__':
+  import sys
+  s = Session()
+  for n,t in zip(*[iter(sys.argv[1:])]*2):
+    print(n,t)
+    print(s.dns(n,t))

dkimpy_milter/util.py

@@ -16,6 +16,40 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
+
+def fold(header):
+    """Fold a header line into multiple crlf-separated lines at column 72.
+    Borrowed from dkimpy and updated to only add \n instead of \r\n because
+    that's what the milter protocol wants.
+
+    >>> text(fold(b'foo'))
+    'foo'
+    >>> text(fold(b'foo  '+b'foo'*24).splitlines()[0])
+    'foo  '
+    >>> text(fold(b'foo'*25).splitlines()[-1])
+    ' foo'
+    >>> len(fold(b'foo'*25).splitlines()[0])
+    72
+    """
+    i = header.rfind(b"\r\n ")
+    if i == -1:
+        pre = b""
+    else:
+        i += 3
+        pre = header[:i]
+        header = header[i:]
+    maxleng = 72
+    while len(header) > maxleng:
+        i = header[:maxleng].rfind(b" ")
+        if i == -1:
+            j = maxleng
+        else:
+            j = i + 1
+        pre += header[:j] + b"\n   "
+        header = header[j:]
+    return pre + header
+
+
 def user_group(userid):
     """Return user and group from UserID"""
     import grp
@@ -31,13 +65,14 @@ def user_group(userid):
     running_gid = grp.getgrnam(gidname).gr_gid
     return running_uid, running_gid
 
+
 def drop_privileges(milterconfig):
     import os
     import syslog
 
     if os.getuid() != 0:
         if milterconfig.get('Syslog'):
-            syslog.syslog('drop_privileges: Not running as root. Cannot drop permissions.')
+            syslog.syslog('drop_privileges: Not root. No action taken.')
         return
 
     # Get user and group
@@ -53,9 +88,9 @@ def drop_privileges(milterconfig):
     # Set umask
     old_umask = os.umask(milterconfig.get('UMask'))
 
-#################
+
 class ExceptHook:
-    def __init__(self, useSyslog = 1, useStderr = 0):
+    def __init__(self, useSyslog=1, useStderr=0):
         self.useSyslog = useSyslog
         self.useStderr = useStderr
 
@@ -71,60 +106,96 @@ class ExceptHook:
                 sys.stderr.write(line)
 
 
-####################
 def setExceptHook():
     import sys
-    sys.excepthook = ExceptHook(useSyslog = 1, useStderr = 1)
+    sys.excepthook = ExceptHook(useSyslog=1, useStderr=1)
+
 
-####################
 def write_pid(milterconfig):
     """Write PID in pidfile.  Will not overwrite an existing file."""
     import os
     import syslog
-    if not os.path.isfile(milterconfig.get('PidFile')):
+    pidfile = milterconfig.get('PidFile')
+    if pidfile is None:
+        return
+    if not os.path.isfile(pidfile):
         pid = str(os.getpid())
         try:
-            f = open(milterconfig.get('PidFile'), 'w')
+            f = open(pidfile, 'w')
         except IOError as e:
+            if str(e)[:35] == '[Errno 2] No such file or directory':
+                piddir = pidfile.rsplit('/', 1)[0]
+                os.mkdir(piddir)
+                user, group = user_group(milterconfig.get('UserID'))
+                os.chown(piddir, user, group)
+                f = open(pidfile, 'w')
                 if milterconfig.get('Syslog'):
-                syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'.format(milterconfig.get('PidFile'), e))
+                    syslog.syslog('PID dir created: {0}'.format(piddir))
+            else:
+                if milterconfig.get('Syslog'):
+                    syslog.syslog('Unable to write pidfle {0}.  IOError: {1}'
+                                  .format(pidfile, e))
                 raise
         f.write(pid)
         f.close()
         user, group = user_group(milterconfig.get('UserID'))
-        os.chown(milterconfig.get('PidFile'), user, group)
+        os.chown(pidfile, user, group)
     else:
         if milterconfig.get('Syslog'):
-            syslog.syslog('Unable to write pidfle {0}.  File exists.'.format(milterconfig.get('PidFile')))
+            syslog.syslog('Unable to write pidfle {0}.  File exists.'
-        raise RuntimeError('Unable to write pidfle {0}.  File exists.'.format(milterconfig.get('PidFile')))
+                          .format(pidfile))
+        raise RuntimeError('Unable to write pidfle {0}.  File exists.'
+                           .format(pidfile))
     return pid
 
-def own_socketfile(milterconfig):
-    """If socket is Unix socket, chown to UserID before dropping privileges"""
-    import os
-    user, group = user_group(milterconfig.get('UserID'))
-    if milterconfig.get('Socket')[:1] == '/':
-        os.chown(milterconfig.get('Socket')[1:], user, group)
-    if milterconfig.get('Socket')[:6] == "local:":
-        os.chown(milterconfig.get('Socket')[6:], user, group)
 
-####################
+def read_keyfile(keyfile, milterconfig):
-def read_keyfile(milterconfig, keytype):
     """Read private key from file."""
     import syslog
-    if keytype == "RSA":
-        keyfile = milterconfig.get('KeyFile')
-    if keytype == "Ed25519":
-        keyfile = milterconfig.get('KeyFileEd25519')
     try:
         f = open(keyfile, 'r')
         keylist = f.readlines()
     except IOError as e:
         if milterconfig.get('Syslog'):
-            syslog.syslog('Unable to read keyfile {0}.  IOError: {1}'.format(keyfile, e))
+            syslog.syslog('Unable to read keyfile {0}.  IOError: {1}'
+                          .format(keyfile, e))
         raise
     f.close()
     key = ''
     for line in keylist:
         key += line
     return key
+
+def read_keytable(tabledict, milterconfig):
+    """Read keytables into in memory configuration data so all keys are read
+    before priviledges are dropped.
+    When loaded, tabeldict is a dict:
+        {searchkey: [donamin, selector, key]}
+    If key is a file (startswith('/'), then the key is returned in its place."""
+    import dkim
+    import syslog
+    for dictkey, values in tabledict.items():
+        if values[-1][:1] == '/' or values[-1][:2] == './' or values[-1][:3] == '../':
+            key = read_keyfile(values[-1], milterconfig)
+        tabledict[dictkey] = [values[0], values[1], key]
+    return tabledict
+
+def get_keys(milterconfig):
+    """Read keys (table or file) into memory before dropping priviledges"""
+    milterconfig['privateRSA'] = False
+    milterconfig['privateRSATable'] = False
+    milterconfig['privateEd25519'] = False
+    milterconfig['privateEd25519Table'] = False
+    if milterconfig.get('KeyTable'):
+        milterconfig['privateRSATable'] = read_keytable(milterconfig.get('KeyTable'),
+                                                        milterconfig)
+    elif milterconfig.get('KeyFile'):
+        milterconfig['privateRSA'] = read_keyfile(milterconfig.get('KeyFile'),
+                                                  milterconfig)
+    if milterconfig.get('KeyTableEd25519'):
+        milterconfig['privateEd25519Table'] = read_keytable(milterconfig.get('KeyTableEd25519'),
+                                                            milterconfig)
+    elif milterconfig.get('KeyFileEd25519'):
+        milterconfig['privateEd25519'] = read_keyfile(milterconfig.get('KeyFileEd25519'),
+                                                      milterconfig)
+    return milterconfig

etc/dkimpy-milter.conf

@@ -1,6 +1,5 @@
 # This is a basic configuration that can easily be adapted to suit a standard
-# installation. For more advanced options, see dkimpy-milter.conf(5) and/or
+# installation. For more advanced options, see dkimpy-milter.conf(5).
-# /usr/share/doc/dkimpy-milter/examples/opendkim.conf.sample.
 
 # Log to syslog
 Syslog			yes
@@ -9,18 +8,16 @@ Syslog			yes
 # privileged user (e.g. Postfix)
 UMask			007
 
-# Sign for example.com with key in /etc/dkimkeys/dkim.key using
+# Sign for example.com with key in /usr/local/etc/dkimkeys/dkim.key using
 # selector '2007' (e.g. 2007._domainkey.example.com)
 #Domain			example.com
-#KeyFile			/etc/mail/dkim.key
+#KeyFile		/usr/local/etc/mail/dkim.key
 #Selector		default
 
 # Commonly-used options; the commented-out versions show the defaults.
 #Canonicalization	relaxed/simple
 #Mode			sv
 
-# Socket local:/var/run/dkimpy-milter/dkimpy-milter.sock
-#
 # ##  Socket socketspec
 # ##
 # ##  Names the socket where this filter should listen for milter connections
@@ -30,15 +27,17 @@ UMask			007
 # ##  inet:port                   to listen on all interfaces
 # ##  local:/path/to/socket       to listen on a UNIX domain socket
 #
-Socket			inet:8892@localhost
+#Socket local:/run/dkimpy-milter/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
 
 ##  PidFile filename
-###      default /var/run/dkimpy-milter/dkimpy-milter.pid
+###      default /run/dkimpy-milter/dkimpy-milter.pid
 ###
 ###  Name of the file where the filter should write its pid before beginning
 ###  normal operations.
 #
-PidFile			/var/run/dkimpy-milter/dkimpy-milter.pid
+PidFile			/run/dkimpy-milter/dkimpy-milter.pid
 
 ##  Userid userid
 ###      default dkimpy-milter

etc/dkimpy-milter.conf.in

@@ -0,0 +1,48 @@
+# This is a basic configuration that can easily be adapted to suit a standard
+# installation. For more advanced options, see dkimpy-milter.conf(5).
+
+# Log to syslog
+Syslog			yes
+
+# Required to use local socket with MTAs that access the socket as a non-
+# privileged user (e.g. Postfix)
+UMask			007
+
+# Sign for example.com with key in @SYSCONFDIR@/dkimkeys/dkim.key using
+# selector '2007' (e.g. 2007._domainkey.example.com)
+#Domain			example.com
+#KeyFile		@SYSCONFDIR@/mail/dkim.key
+#Selector		default
+
+# Commonly-used options; the commented-out versions show the defaults.
+#Canonicalization	relaxed/simple
+#Mode			sv
+
+# ##  Socket socketspec
+# ##
+# ##  Names the socket where this filter should listen for milter connections
+# ##  from the MTA.  Required.  Should be in one of these forms:
+# ##
+# ##  inet:port@address           to listen on a specific interface
+# ##  inet:port                   to listen on all interfaces
+# ##  local:/path/to/socket       to listen on a UNIX domain socket
+#
+#Socket local:@RUNSTATEDIR@/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
+
+##  PidFile filename
+###      default /run/dkimpy-milter/dkimpy-milter.pid
+###
+###  Name of the file where the filter should write its pid before beginning
+###  normal operations.
+#
+PidFile			@RUNSTATEDIR@/dkimpy-milter.pid
+
+##  Userid userid
+###      default dkimpy-milter
+###
+###  Change to user "userid" before starting normal operation?  May include
+###  a group ID as well, separated from the userid by a colon.
+#
+UserID			dkimpy-milter

man/dkimpy-milter.8

@@ -127,7 +127,6 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.IX Title "dkimpy-milter 8"
 .TH dkimpyy-milter 8
 .SH NAME
 .B dkimpy
@@ -156,7 +155,7 @@ values, or to a file that contains them, or a database containing the data.
 
 Some data sets require that the value contain more than one entry.  How this
 is done depends on which data set type is used.  Not all these datasets are
-currently used by dkimp-milter.  See
+currently used by dkimpy-milter.  See
 .B dkimpy-milter.conf(5)
 for details about specific options and which dataset types they use.
 
@@ -169,7 +168,7 @@ one per line.  If a line contains whitespace-separated values, then the
 line is presumed to define a key and its corresponding value.  Blank lines
 are ignored, and the hash ("#") character denotes the start of a comment.
 If a value contains multiple entries, the entries should be separated by
-colons. [Not implemented yet]
+colons.
 .TP
 .I b)
 If the string begins with "refile:", then the remainder of the string is
@@ -179,28 +178,18 @@ to the first whitespace, and the portion after that whitespace is taken as
 the value to be used when that pattern is matched.  Patterns are simple
 wildcard patterns, matching all text except that the asterisk ("*") character
 is considered a wildcard.  If a value contains multiple entries, the entries
-should be separated by colons. [Not implemented yet]
+should be separated by colons.
-.TP
-.I c)
-If the string begins with "db:" and the program was compiled with
-Sleepycat DB support, then the remainder of the string is presumed to
-identify a Sleepycat database containing keys and corresponding values.
-These may be used only to test for membership in the data set, or for
-storing keys and corresponding values.  If a value contains multiple entries,
-the entries should be separated by colons. [Not implemented yet]
-.TP
-.I h)
-If the string contains none of these prefixes but ends with ".db", it
-is presumed to be a Sleepycat DB as described above (if support for same
-is compiled in).  [Not implemented yet]
 .TP
 .I i)
 If the string contains none of these prefixes but starts with a slash ("/")
-character, it is presumed to be a flat file as described above.  [Not implemented yet]
+character, or "./" or "../", it is presumed to be a flat file as described
+above. Note: In OpenDKIM "./" and "../" only apply to KeyTable, but for
+dkimpy-milter it is generally applicable and KeyTable specification is not
+a special case.
 .TP
 .I j)
 If the string begins with "csl:", the string is treated as a comma-separated
-list as described in m) below.  [Not implemented yet]
+list as described in m) below.
 .TP
 .I l)
 If the string begins with "mdb:", it refers to a directory that contains
@@ -214,9 +203,10 @@ pairs as described above.
 .SH OPTIONS
 .TP
 See
-.I dkimpy-milter.conf(5)
+.I dkimpy-milter.conf (5)
-information about available options.  Unlike OpenDKIM, dkimpy-milter does not
+for information about available options.  Unlike OpenDKIM, with the exception of
-support command line option switches.
+\-P for the pidfile and specifying the configuration file to use,
+dkimpy-milter does not support command line option switches.
 
 When signing a message, a
 .I DKIM-Signature:
@@ -279,7 +269,7 @@ proposal, and Cisco's
 .B Internet Identified Mail
 (IIM) proposal.
 .SH VERSION
-This man page covers version 0.9.2 of
+This man page covers version 1.1.0 of
 .I dkimpy-milter.
 .SH COPYRIGHT
 Copyright (c) 2005-2008, Sendmail, Inc. and its suppliers.  All rights
@@ -288,7 +278,7 @@ reserved.
 Copyright (c) 2009-2013, 2015, The Trusted Domain Project.
 All rights reserved.
 
-Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>
+Copyright (c) 2018, 2019 Scott Kitterman <scott@kitterman.com>
 .SH SEE ALSO
 .I dkimpy-milter.conf(5), sendmail(8)
 .P
@@ -302,4 +292,6 @@ RFC6376 - DomainKeys Identified Mail
 .P
 RFC7601 - Message Header Field for Indicating Message Authentication Status
 .P
-draft-ietf-dcrup-dkim-crypto - A new cryptographic signature method for DKIM
+RFC8301 - Cryptographic Algorithm and Key Usage Update to DomainKeys Identified Mail (DKIM)
+.P
+RFC8463 - A New Cryptographic Signature Method for DomainKeys Identified Mail (DKIM)

man/dkimpy-milter.conf.5

@@ -127,16 +127,13 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.IX Title "dkimpy-milter.conf 5"
+.TH dkimpy-milter.conf 5 "2019-04-25"
-.TH dkimpy-milter.conf 5 "2018-02-12"
 .SH "NAME"
 dkimpy-milter \- Python milter for DKIM signing and validation
 .SH "VERSION"
-.IX Header "VERSION"
+1\.2\.0
-0\.9\.2
 
 .SH "DESCRIPTION"
-.IX Header "DESCRIPTION"
 .I dkimpy-milter(8)
 implements the
 .B DKIM
@@ -155,23 +152,21 @@ the value is processed.  For positive values, the following are accepted:
 "F", "f", "N", "n", "0".
 
 The provided setup.py installs this configuration file in /etc or
-/usr/local/etc.
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
 
 Command line invocation of parameters as is done by OpenDKIM is not supported.
 
 .SH "USAGE"
-.IX Header "USAGE"
 Usage:
-  dkimpy-milter [/etc/dkimpy-milter.conf]
+  dkimpy-milter [/usr/local/etc/dkimpy-milter/dkimpy-milter.conf]
 
 .SH "OTHER DOCUMENTATION"
-.IX Header "OTHER DOCUMENTATION"
 This documentation assumes you have read Postfix's README_FILES/MILTER_README
 (or Sendmail equivalent) and are generally familiar with Domain Keys Identified
 Mail (DKIM).  See RFC 6376 for details.
 
 .SH "SYNOPSIS"
-.IX Header "SYNOPSIS"
 
 dkimpy-milter operates with a default installed configuration file and 
 set of default configuration options that are used if the configuration file
@@ -181,14 +176,12 @@ files can be used directly.  Not all OpenDKIM options are supported.  If an
 unsupported option from OpenDKIM is specified, an error will be raised.
 
 .SH "DESCRIPTION"
-.IX Header "DESCRIPTION"
 
 Configuration options are described here and in the configuration file 
 provided with the package.  The provided setup.py installs this configuration 
 file in /etc or /usr/local/etc.
 
 .SH "OPTIONS"
-.IX Header "OPTIONS"
 
 .TP
 .I AuthservID (string)
@@ -215,18 +208,14 @@ The value may include two different canonicalizations separated by a
 slash ("/") character, in which case the first will be applied to the
 header and the second to the body.
 
-.TP
-.I Diagnostics (Boolean)
-Requests the inclusion of "z=" tags in signatures, which encode the
-original header field set for use by verifiers when diagnosing verification
-failures.  Not recommended for normal operation. [dkimpy-milter specific: also
-increases the verbosity of Syslog logging if enabled.]
-
 .TP
 .I DiagnosticDirectory (string)
 Directory into which to write diagnostic reports when message verification
-fails on a message bearing a "z=" tag.  If not set (the default), these files
+fails.  If not set (the default), these files are not generated.  The
-are not generated.
+directory must exist, dkimpy-milter will not create it and an error will be
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
 
 .TP
 .I Domain (dataset)
@@ -236,11 +225,13 @@ domains will be verified rather than being signed.
 This parameter is not required if a
 .I SigningTable
 is in use; in that case, the list of signed domains is implied by the
-lines in that file. [SigningTable NOT IMPLEMENTED]
+lines in that file.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+or
+.I KeyTableD25119
+is defined.
 
 .TP
 .I InternalHosts (dataset)
@@ -253,20 +244,81 @@ Naturally, providing a value here overrides the default, so if mail from
 127.0.0.1 should be signed, the list provided here should include that
 address explicitly. [PeerList NOT IMPLEMENTED]
 
+Mail sent via connections from InternalHosts will not have any existing DKIM
+signatures verified.  This is not overridden by MacroList or Mode.  If the
+Mode is 'v', then no actions will be performed.
+
 .TP
 .I KeyFile (string)
 Gives the location of a PEM-formatted private key to be used for RSA signing
 all messages.  Ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
-.I KeyFileEd25119 (string)
+.I KeyFileEd25519 (string)
 Gives the location of a Ed25519 private key to be used for Ed25519 signing
 all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
 generation (as used in dkimpy).  Ignored if a 
 .I KeyTableEd25519
-is defined.  [KeyTableEd25519 NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I MacroList (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be a
+local user and therefore whether or not the message should be signed.  If
+a
+.I value
+is specified matching a macro name in the data set, the value of the macro
+must match a value specified (matching is case-sensitive), otherwise the
+macro must be defined but may contain any value.  The set is empty by
+default, meaning macros are not considered when making the sign-verify
+decision.  The general format of the value is
+.I value1[|value2[|...]];
+if one or more value is defined then the macro must be set to one of the
+listed values, otherwise the macro must be set but can contain any
+value.
+
+In order for the macro and its value to be available to the filter for
+checking, the MTA must send it during the protocol exchange.  This is either
+accomplished via manual configuration of the MTA to send the desired macros
+or, for MTA/filter combinations that support the feature, the filter can
+request those macros that are of interest.  The latter is a feature negotiated
+at the time the filter receives a connection from the MTA and its availability
+depends upon the version of milter used to compile the filter and the version
+of the MTA making the connection.
+
+Mail sent via connections where macros that are in MacroList are provided
+will not have any existing DKIM signatures verified.  If the Mode is 'v', then
+no actions will be performed.
+
+.TP
+.I MacroListVerify (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be an
+external source and therefore whether or not the message should be signed.
+Entries in this data set follow the same form as those of the
+.I MacroList
+option above.  [this option is not inhereted from OpenDKIM]  
+
+Mail sent via connections where macros that are in MacroListVerify are
+provided will be not DKIM signed.  If the Mode is 's', then no actions will
+be performed.
 
 .TP
 .I Mode (string)
@@ -286,7 +338,48 @@ When signing mode is enabled, one of the following combinations must also
 be set:
 (a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
 (b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
-[fooTable options NOT IMPLEMENTED]
+
+The action to sign or verify is also affected by the InternalHosts, MacroList,
+and MacroListVerify options.  Those options may preclude signing or
+verification in some cases, but will not enable signing or verifying if not
+allowed by Mode.
+
+.TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
+
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
 
 .TP
 .I PeerList (dataset)
@@ -308,7 +401,7 @@ will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
 .TP
 .I PidFile (string)
 Specifies the path to a file that should be created at process start
-containing the process ID.
+containing the process ID.  If not specified, no such file will be created.
 
 .TP
 .I Selector (string)
@@ -322,7 +415,7 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTable
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
 
 .TP
 .I SelectorEd25519 (string)
@@ -336,7 +429,33 @@ parameter below for more information.
 
 This parameter is ignored if a
 .I KeyTableEd25519
-is defined. [KeyTable NOT IMPLEMENTED]
+is defined.
+
+.TP
+.I SignHeaders (dataset)
+Specifies the set of header fields that should be included when generating
+signatures.  If the list omits any header field that is mandated by the DKIM
+specification, those fields are implicitly added.  By default, those fields
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
+will be signed by the filter.  See the
+.I OmitHeaders
+configuration option for more information about the format and interpretation
+of this field.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select one or more signing identities to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
 
 .TP
 .I Socket (string)
@@ -361,6 +480,12 @@ is not given as either a hostname or an IP address, the socket will be
 listening on all interfaces.  A literal IP address must be enclosed in
 square brackets.  This option is mandatory in the configuration file.
 
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
 .TP
 .I Syslog (Boolean)
 Log via calls to
@@ -374,7 +499,7 @@ Log via calls to
 using the named facility.  The facility names are the same as the ones
 allowed in
 .I syslog.conf(5).
-The default is "mail".  [Hardcoded to default for now]
+The default is "mail".
 
 .TP
 .I SyslogSuccess (Boolean)
@@ -409,12 +534,32 @@ unless an alternate
 .I group
 is specified.
 
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
+Default location of this file.
+
 .SH "AUTHORS"
-.IX Header "AUTHORS"
 \ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
-It is based on dkimpy-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
 Copyright (c) 2013-2015 Stuart D. Gathman
-Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>.  
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
 .PP
 This man-page was created by Scott Kitterman <scott@kitterman.com>.
 
@@ -428,4 +573,4 @@ See LICENSE.
 
 Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
 of the package.
-Copyright (c) 2018, Scott Kitterman <scott@kitterman.com>
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

man/dkimpy-milter.conf.5.in

@@ -0,0 +1,576 @@
+\"
+.\" Standard preamble:
+.\" ========================================================================
+.de Sh \" Subsection heading
+.br
+.if t .Sp
+.ne 5
+.PP
+\fB\\$1\fR
+.PP
+..
+.de Sp \" Vertical space (when we can't use .PP)
+.if t .sp .5v
+.if n .sp
+..
+.de Vb \" Begin verbatim text
+.ft CW
+.nf
+.ne \\$1
+..
+.de Ve \" End verbatim text
+.ft R
+.fi
+..
+.\" Set up some character translations and predefined strings.  \*(-- will
+.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
+.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
+.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
+.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
+.\" nothing in troff, for use with C<>.
+.tr \(*W-
+.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
+.ie n \{\
+.    ds -- \(*W-
+.    ds PI pi
+.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
+.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
+.    ds L" ""
+.    ds R" ""
+.    ds C` ""
+.    ds C' ""
+'br\}
+.el\{\
+.    ds -- \|\(em\|
+.    ds PI \(*p
+.    ds L" ``
+.    ds R" ''
+'br\}
+.\"
+.\" If the F register is turned on, we'll generate index entries on stderr for
+.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index
+.\" entries marked with X<> in POD.  Of course, you'll have to process the
+.\" output yourself in some meaningful fashion.
+.if \nF \{\
+.    de IX
+.    tm Index:\\$1\t\\n%\t"\\$2"
+..
+.    nr % 0
+.    rr F
+.\}
+.\"
+.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
+.\" way too many mistakes in technical documents.
+.hy 0
+.if n .na
+.\"
+.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
+.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
+.    \" fudge factors for nroff and troff
+.if n \{\
+.    ds #H 0
+.    ds #V .8m
+.    ds #F .3m
+.    ds #[ \f1
+.    ds #] \fP
+.\}
+.if t \{\
+.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
+.    ds #V .6m
+.    ds #F 0
+.    ds #[ \&
+.    ds #] \&
+.\}
+.    \" simple accents for nroff and troff
+.if n \{\
+.    ds ' \&
+.    ds ` \&
+.    ds ^ \&
+.    ds , \&
+.    ds ~ ~
+.    ds /
+.\}
+.if t \{\
+.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
+.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
+.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
+.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
+.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
+.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
+.\}
+.    \" troff and (daisy-wheel) nroff accents
+.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
+.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
+.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
+.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
+.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
+.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
+.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
+.ds ae a\h'-(\w'a'u*4/10)'e
+.ds Ae A\h'-(\w'A'u*4/10)'E
+.    \" corrections for vroff
+.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
+.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
+.    \" for low resolution devices (crt and lpr)
+.if \n(.H>23 .if \n(.V>19 \
+\{\
+.    ds : e
+.    ds 8 ss
+.    ds o a
+.    ds d- d\h'-1'\(ga
+.    ds D- D\h'-1'\(hy
+.    ds th \o'bp'
+.    ds Th \o'LP'
+.    ds ae ae
+.    ds Ae AE
+.\}
+.rm #[ #] #H #V #F C
+.\" ========================================================================
+.\"
+.TH dkimpy-milter.conf 5 "2019-04-25"
+.SH "NAME"
+dkimpy-milter \- Python milter for DKIM signing and validation
+.SH "VERSION"
+1\.2\.0
+
+.SH "DESCRIPTION"
+.I dkimpy-milter(8)
+implements the
+.B DKIM
+specification for signing and verifying e-mail messages on a per-domain
+basis.  This file is its configuration file.
+
+Blank lines are ignored.  Lines containing a hash ("#") character are
+truncated at the hash character to allow for comments in the file.
+
+Other content should be the name of a parameter, followed by white space,
+followed by the value of that parameter, each on a separate line.
+
+For parameters that are Boolean in nature, only the first byte of
+the value is processed.  For positive values, the following are accepted:
+"T", "t", "Y", "y", "1".  For negative values, the following are accepted:
+"F", "f", "N", "n", "0".
+
+The provided setup.py installs this configuration file in /etc or
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
+
+Command line invocation of parameters as is done by OpenDKIM is not supported.
+
+.SH "USAGE"
+Usage:
+  dkimpy-milter [@CONFDIR@/dkimpy-milter.conf]
+
+.SH "OTHER DOCUMENTATION"
+This documentation assumes you have read Postfix's README_FILES/MILTER_README
+(or Sendmail equivalent) and are generally familiar with Domain Keys Identified
+Mail (DKIM).  See RFC 6376 for details.
+
+.SH "SYNOPSIS"
+
+dkimpy-milter operates with a default installed configuration file and 
+set of default configuration options that are used if the configuration file
+cannot be found.  These options can be changed by changing the installed 
+configuration files.  For users transitioning from OpenDKIM, OpenDKIM config
+files can be used directly.  Not all OpenDKIM options are supported.  If an
+unsupported option from OpenDKIM is specified, an error will be raised.
+
+.SH "DESCRIPTION"
+
+Configuration options are described here and in the configuration file 
+provided with the package.  The provided setup.py installs this configuration 
+file in /etc or /usr/local/etc.
+
+.SH "OPTIONS"
+
+.TP
+.I AuthservID (string)
+Sets the "authserv-id" to use when generating the Authentication-Results:
+header field after verifying a message.  The default is to use the name of
+the MTA processing the message.  If the string "HOSTNAME" is provided, the
+name of the host running the filter (as returned by the
+.I gethostname(3)
+function) will be used.
+
+.TP
+.I Canonicalization (string)
+Selects the canonicalization method(s) to be used when signing messages.
+When verifying, the message's DKIM-Signature: header field specifies
+the canonicalization method.  The recognized values are
+.I relaxed
+and
+.I simple
+as defined by the DKIM specification.  The default is
+.I relaxed
+/
+.I simple.
+The value may include two different canonicalizations separated by a
+slash ("/") character, in which case the first will be applied to the
+header and the second to the body.
+
+.TP
+.I DiagnosticDirectory (string)
+Directory into which to write diagnostic reports when message verification
+fails.  If not set (the default), these files are not generated.  The
+directory must exist, dkimpy-milter will not create it and an error will be
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
+
+.TP
+.I Domain (dataset)
+A set of domains whose mail should be signed by this filter.  Mail from other
+domains will be verified rather than being signed.
+
+This parameter is not required if a
+.I SigningTable
+is in use; in that case, the list of signed domains is implied by the
+lines in that file.
+
+This parameter is ignored if a
+.I KeyTable
+or
+.I KeyTableD25119
+is defined.
+
+.TP
+.I InternalHosts (dataset)
+Identifies a set internal hosts whose mail should be signed rather
+than verified.  Entries in this data set follow the same form as those of
+the
+.I PeerList
+option below.  If not specified, the default of "127.0.0.1" is applied.
+Naturally, providing a value here overrides the default, so if mail from
+127.0.0.1 should be signed, the list provided here should include that
+address explicitly. [PeerList NOT IMPLEMENTED]
+
+Mail sent via connections from InternalHosts will not have any existing DKIM
+signatures verified.  This is not overridden by MacroList or Mode.  If the
+Mode is 'v', then no actions will be performed.
+
+.TP
+.I KeyFile (string)
+Gives the location of a PEM-formatted private key to be used for RSA signing
+all messages.  Ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I KeyFileEd25519 (string)
+Gives the location of a Ed25519 private key to be used for Ed25519 signing
+all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
+generation (as used in dkimpy).  Ignored if a 
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I MacroList (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be a
+local user and therefore whether or not the message should be signed.  If
+a
+.I value
+is specified matching a macro name in the data set, the value of the macro
+must match a value specified (matching is case-sensitive), otherwise the
+macro must be defined but may contain any value.  The set is empty by
+default, meaning macros are not considered when making the sign-verify
+decision.  The general format of the value is
+.I value1[|value2[|...]];
+if one or more value is defined then the macro must be set to one of the
+listed values, otherwise the macro must be set but can contain any
+value.
+
+In order for the macro and its value to be available to the filter for
+checking, the MTA must send it during the protocol exchange.  This is either
+accomplished via manual configuration of the MTA to send the desired macros
+or, for MTA/filter combinations that support the feature, the filter can
+request those macros that are of interest.  The latter is a feature negotiated
+at the time the filter receives a connection from the MTA and its availability
+depends upon the version of milter used to compile the filter and the version
+of the MTA making the connection.
+
+Mail sent via connections where macros that are in MacroList are provided
+will not have any existing DKIM signatures verified.  If the Mode is 'v', then
+no actions will be performed.
+
+.TP
+.I MacroListVerify (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be an
+external source and therefore whether or not the message should be signed.
+Entries in this data set follow the same form as those of the
+.I MacroList
+option above.  [this option is not inhereted from OpenDKIM]  
+
+Mail sent via connections where macros that are in MacroListVerify are
+provided will be not DKIM signed.  If the Mode is 's', then no actions will
+be performed.
+
+.TP
+.I Mode (string)
+Selects operating modes.  The string is a concatenation of characters that
+indicate which mode(s) of operation are desired.  Valid modes are
+.I s
+(signer) and
+.I v
+(verifier).  The default is
+.I sv
+except in test mode (see the
+.I opendkim(8)
+man page)
+in which case the default is
+.I v.
+When signing mode is enabled, one of the following combinations must also
+be set:
+(a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
+(b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
+
+The action to sign or verify is also affected by the InternalHosts, MacroList,
+and MacroListVerify options.  Those options may preclude signing or
+verification in some cases, but will not enable signing or verifying if not
+allowed by Mode.
+
+.TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
+
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
+
+.TP
+.I PeerList (dataset)
+Identifies a set of "peers" that identifies clients whose connections
+should be accepted without processing by this filter.  The set
+should contain on each line a hostname, domain name (e.g. ".example.com"),
+IP address, an IPv6 address (including an IPv4 mapped address), or a
+CIDR-style IP specification (e.g. "192.168.1.0/24").  An entry beginning
+with a bang ("!") character means "not", allowing exclusions of specific
+hosts that are otherwise members of larger sets.  Host and domain names are 
+matched first, then the IP or IPv6 address depending on the connection 
+type.  More precise entries are preferred over less precise ones, i.e. 
+"192.168.1.1" will match before "!192.168.1.0/24".  The text form of IPv6 
+addresses will be forced to lowercase when queried (RFC5952), so the contents
+of this data set should also use lowercase.  The IP address portion of an
+entry may optionally contain square brackets; both forms (with and without)
+will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
+
+.TP
+.I PidFile (string)
+Specifies the path to a file that should be created at process start
+containing the process ID.  If not specified, no such file will be created.
+
+.TP
+.I Selector (string)
+Defines the name of the selector to be used when signing messages using RSA.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I SelectorEd25519 (string)
+Defines the name of the selector to be used when signing messages using Ed25519.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I SignHeaders (dataset)
+Specifies the set of header fields that should be included when generating
+signatures.  If the list omits any header field that is mandated by the DKIM
+specification, those fields are implicitly added.  By default, those fields
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
+will be signed by the filter.  See the
+.I OmitHeaders
+configuration option for more information about the format and interpretation
+of this field.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select a signing identity to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I Socket (string)
+Specifies the socket that should be established by the filter to receive
+connections from
+.I postfix(1)
+in order to provide service.
+.I socketspec
+is in one of two forms:
+.I local:path,
+which creates a UNIX domain socket at the specified
+.I path,
+or
+.I inet:port[@host]
+or
+.I inet6:port[@host]
+which creates a TCP socket on the specified
+.I port
+and in the specified protocol family.  If the
+.I host
+is not given as either a hostname or an IP address, the socket will be
+listening on all interfaces.  A literal IP address must be enclosed in
+square brackets.  This option is mandatory in the configuration file.
+
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
+.TP
+.I Syslog (Boolean)
+Log via calls to
+.I syslog(3)
+any interesting activity.
+
+.TP
+.I SyslogFacility (string)
+Log via calls to
+.I syslog(3)
+using the named facility.  The facility names are the same as the ones
+allowed in
+.I syslog.conf(5).
+The default is "mail".
+
+.TP
+.I SyslogSuccess (Boolean)
+Log via calls to
+.I syslog(3)
+additional entries indicating successful signing or verification of
+messages.
+
+.TP
+.I UMask (integer)
+Requests a specific permissions mask to be used for file creation.
+This only really applies to creation of the socket when
+.I Socket
+specifies a UNIX domain socket, and to the
+.I PidFile
+(if any); temporary files are created by the
+.I mkstemp(3)
+function that enforces a specific file mode on creation regardless
+of the process umask.  See
+.I umask(2)
+for more information.
+
+.TP
+.I UserID (string)
+Attempts to become the specified userid before starting operations.
+The value is of the form
+.I userid[:group].
+The process will be assigned all of the groups and primary group ID of
+the named
+.I userid
+unless an alternate
+.I group
+is specified.
+
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I @CONFDIR@/dkimpy-milter.conf
+Default location of this file.
+
+.SH "AUTHORS"
+\ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+Copyright (c) 2013-2015 Stuart D. Gathman
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
+.PP
+This man-page was created by Scott Kitterman <scott@kitterman.com>.
+
+.SH COPYRIGHT
+Configuration items derived from OpenDKIM 2.11.0 opendkim.conf.5.in:
+Copyright (c) 2007, 2008, Sendmail, Inc. and its suppliers.  All rights
+reserved.  See LICENSE.Sendmail.
+
+Copyright (c) 2009-2015, The Trusted Domain Project.  All rights reserved.
+See LICENSE.
+
+Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
+of the package.
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

setup.py

@@ -1,7 +1,7 @@
-#! /usr/bin/python
+#! /usr/bin/python3
 # dkimpy-milter: A DKIM signing/verification Milter application
 # Author: Scott Kitterman <scott@kitterman.com>
-# Copyright 2018 Scott Kitterman
+# Copyright 2018,2019 Scott Kitterman
 """    This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; either version 2 of the License, or
@@ -17,27 +17,94 @@
     51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA."""
 
 from setuptools import setup
+import distutils.cmd
+import distutils.log
+import sys
 import os
-import dkimpy_milter
+import subprocess
 
 description = "Domain Keys Identified Mail (DKIM) signing/verifying milter for Postfix/Sendmail."
 
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+
+class FileMacroExpand(distutils.cmd.Command):
+    description = "Expand @@ variables in input files, simlar to make macros."
+    user_options = [
+        ('sysconfigdir=', 'e', 'Specify system configuration directory. [/usr/local/etc]'),
+        ('sbindir=', 's', 'Specify system binary directory. [/usr/local/sbin]'),
+        ('bindir=', 'b', 'Specify binary directory. [/usr/loca/bin]'),
+        ('rundir=', 'r', 'Specify run state directory. [/run]'),
+    ]
+
+    def initialize_options(self):
+        self.sysconfigdir = '/usr/local/etc'
+        self.sbindir = '/usr/local/sbin'
+        self.bindir = '/usr/local/bin'
+        self.rundir = '/run'
+
+    def finalize_options(self):
+        self.configdir = self.sysconfigdir + '/dkimpy-milter'
+        self.rundir += '/dkimpy-milter'
+
+    def run(self):
+        files = ['etc/dkimpy-milter.conf', 'man/dkimpy-milter.conf.5', \
+                 'system/dkimpy-milter.service', 'system/dkimpy-milter', \
+                 'system/dkimpy-milter.openrc', \
+                 'system/socket-activation/dkimpy-milter.service', \
+                 'system/socket-activation/dkimpy-milter.socket', ]
+        for infile in files:
+            outfile = ''
+            try:
+                filein = open(infile + '.in')
+                for line in filein:
+                    for function in ["@SYSCONFDIR@", "@CONFDIR@", "@SBINDIR@", "@BINDIR@", "@RUNSTATEDIR@"]:
+                        splitline = line.split(function)
+                        if len(splitline) > 1:
+                            if function == "@SYSCONFDIR@":
+                                line = splitline[0] + self.sysconfigdir + splitline[1]
+                            elif function == "@CONFDIR@":
+                                line = splitline[0] + self.configdir + splitline[1]
+                            elif function == "@SBINDIR@":
+                                line = splitline[0] + self.sbindir + splitline[1]
+                            elif function == "@BINDIR@":
+                                line = splitline[0] + self.bindir + splitline[1]
+                            elif function == "@RUNSTATEDIR@":
+                                line = splitline[0] + self.rundir + splitline[1]
+                    outfile += line
+                out = open(infile, 'w')
+                for line in outfile:
+                    out.write(line)
+                out.close()
+            except FileNotFoundError as x:
+                pass
+
+kw = {}  # Work-around for lack of 'or' requires in setuptools.
+try:
+    import dns
+    kw['install_requires'] = ['dkimpy>=1.1.0', 'pymilter>=1.0.5', 'authres>=1.1.0', 'PyNaCl', 'dnspython>=1.16.0']
+except ImportError:  # If PyDNS is not installed, prefer dnspython
+    kw['install_requires'] = ['dkimpy>=1.1.0', 'pymilter>=1.0.5', 'authres>=1.1.0', 'PyNaCl', 'Py3DNS']
+
 setup(
     name='dkimpy-milter',
-    version=dkimpy_milter.__version__,
+    version='1.2.3',
     author='Scott Kitterman',
     author_email='scott@kitterman.com',
     url='https://launchpad.net/dkimpy-milter',
     description=description,
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     download_url = "https://pypi.python.org/pypi/dkimpy-milter",
     classifiers= [
-        'Development Status :: 3 - Alpha',
+        'Development Status :: 5 - Production/Stable',
         'Environment :: No Input/Output (Daemon)',
         'Intended Audience :: System Administrators',
         'License :: OSI Approved :: GNU General Public License (GPL)',
         'Natural Language :: English',
         'Operating System :: POSIX',
-        'Programming Language :: Python :: 2 :: Only',
+        'Programming Language :: Python :: 3 :: Only',
         'Topic :: Communications :: Email :: Mail Transport Agents',
         'Topic :: Communications :: Email :: Filters',
         'Topic :: Security',
@@ -51,10 +118,14 @@ setup(
     include_package_data=True,
     data_files=[(os.path.join('share', 'man', 'man5'),
         ['man/dkimpy-milter.conf.5']), (os.path.join('share', 'man', 'man8'),
-        ['man/dkimpy-milter.8']), ('etc', ['etc/dkimpy-milter.conf']),
+        ['man/dkimpy-milter.8']), (os.path.join('etc', 'dkimpy-milter'),
-        (os.path.join('/lib', 'systemd', 'system'),
+        ['etc/dkimpy-milter.conf']), (os.path.join('lib', 'systemd', 'system'),
-        ['system/dkimpy-milter.service']),(os.path.join('/etc', 'init.d'),
+        ['system/dkimpy-milter.service']),(os.path.join('etc', 'init.d'),
-        ['system/dkimpy-milter'])],
+        ['system/dkimpy-milter']), (os.path.join('etc', 'init.d'),
-    install_requires = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl'],
+        ['system/dkimpy-milter.openrc'])],
     zip_safe = False,
+    cmdclass={
+        'expand': FileMacroExpand,
+    },
+    **kw
 )

system/dkimpy-milter

@@ -13,32 +13,28 @@
 # Provides:          dkim-milter dkim-milter-python dkimpy-milter
 # Required-Start:    $remote_fs $syslog $network $time
 # Required-Stop:     $remote_fs $syslog $network
-# Should-Start:
-# Should-Stop:
 # Default-Start:     2 3 4 5
 # Default-Stop:      0 1 6
 # Short-Description: dkimpy-milter
 # Description:       Python DKIM Milter for Sendmail and Postfix
 ### END INIT INFO
-prefix="/usr/local"
+sysconfdir="/usr/local/etc/dkimpy-milter"
-exec_prefix=${prefix}
+bindir="/usr/local/bin"
-sysconfdir="/etc/dkimpy-milter"
+RUNDIR="/run/dkimpy-milter"
-bindir="${exec_prefix}/bin/"
-RUNDIR="/var/run/dkimpy-milter"
 DAEMON=${bindir}/dkimpy-milter
 PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
 NAME=dkimpy-milter
 DESC="Python DKIM Milter"
 USER=dkimpy-milter
 GROUP=dkimpy-milter
-SOCKET=$RUNDIR/dkimpy-milter.pid
+SOCKET=$RUNDIR/dkimpy-milter.sock
 
 test -x $DAEMON || exit 0
 
 # Include dkimpy-python defaults if available
 # Typically not used
-if [ -f /etc/default/dkimpy-milter ] ; then
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
-	. /etc/default/dkimpy-milter
+	. $sysconfdir/default/dkimpy-milter
 fi
 
 set -e
@@ -69,14 +65,14 @@ case "$1" in
 			fi
 		fi
 	fi
-
+        start-stop-daemon --start --background --quiet --pidfile \
-	start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
                 $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
 	echo "$NAME."
 	;;
   stop)
 	echo -n "Stopping $DESC: "
 	if [ -f $RUNDIR/$NAME.pid ]; then
+		chown root:root $RUNDIR/$NAME.pid
 		start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid 
 		rm $RUNDIR/$NAME.pid
 		#echo $SOCKET
@@ -89,6 +85,7 @@ case "$1" in
   force-reload)
         echo -n "Force reloading $DESC: "
         if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
                 start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
                 rm $RUNDIR/$NAME.pid
                 #echo $SOCKET
@@ -97,7 +94,7 @@ case "$1" in
                 fi
         fi
         sleep 1
-        start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+        start-stop-daemon --start --background --quiet --pidfile \
                 $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
         echo "$NAME."
         ;;
@@ -105,6 +102,7 @@ case "$1" in
         echo "Restarting $DESC: "
         echo -n "Stopping $DESC: "
         if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
                 start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
                 rm $RUNDIR/$NAME.pid
                 #echo $SOCKET
@@ -115,12 +113,12 @@ case "$1" in
         echo "$NAME."
 	sleep 1
         echo -n "Starting $DESC: "
-        start-stop-daemon --start --chuid $USER --background --quiet --pidfile \
+        start-stop-daemon --start --background --quiet --pidfile \
                 $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
         echo "$NAME."
 	;;
   status)
-        status_of_proc -p /var/run/dkimpy-milter/dkimpy-milter.pid /usr/local/bin/dkimpy-milter dkimpy-milter
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
         ;;
 
   *)

system/dkimpy-milter.in

@@ -0,0 +1,131 @@
+#! /bin/sh
+#
+# skeleton	example file to build /etc/init.d/ scripts.
+#		This file should be used to construct scripts for /etc/init.d.
+#
+#		Written by Miquel van Smoorenburg <miquels@cistron.nl>.
+#		Modified for Debian 
+#		by Ian Murdock <imurdock@gnu.ai.mit.edu>.
+#
+# Version:	@(#)skeleton  1.9  26-Feb-2001  miquels@cistron.nl
+#
+### BEGIN INIT INFO
+# Provides:          dkim-milter dkim-milter-python dkimpy-milter
+# Required-Start:    $remote_fs $syslog $network $time
+# Required-Stop:     $remote_fs $syslog $network
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: dkimpy-milter
+# Description:       Python DKIM Milter for Sendmail and Postfix
+### END INIT INFO
+sysconfdir="@CONFDIR@"
+bindir="@BINDIR@"
+RUNDIR="@RUNSTATEDIR@"
+DAEMON=${bindir}/dkimpy-milter
+PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
+NAME=dkimpy-milter
+DESC="Python DKIM Milter"
+USER=dkimpy-milter
+GROUP=dkimpy-milter
+SOCKET=$RUNDIR/dkimpy-milter.sock
+
+test -x $DAEMON || exit 0
+
+# Include dkimpy-python defaults if available
+# Typically not used
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
+	. $sysconfdir/default/dkimpy-milter
+fi
+
+set -e
+
+. /lib/lsb/init-functions
+
+case "$1" in
+  start)
+	echo -n "Starting $DESC: "
+	# Create the run directory if it doesn't exist
+	if [ ! -d $RUNDIR ]; then
+		install -o $USER -g $GROUP -m 755 -d $RUNDIR || return 2
+	fi
+
+	# Clean up stale sockets
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		pid=`cat $RUNDIR/$NAME.pid`
+		if ! ps -C $DAEMON -s $pid >/dev/null; then
+			rm $RUNDIR/$NAME.pid
+			# UNIX sockets may be specified with or without the
+			# local: prefix; handle both
+			t=`echo $SOCKET | cut -d: -f1`
+			s=`echo $SOCKET | cut -d: -f2`
+			if [ -e $s -a -S $s ]; then
+				if [ "$t" = "$s" -o "$t" = "local" ]; then
+					rm $s
+				fi
+			fi
+		fi
+	fi
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+	echo "$NAME."
+	;;
+  stop)
+	echo -n "Stopping $DESC: "
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		chown root:root $RUNDIR/$NAME.pid
+		start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid 
+		rm $RUNDIR/$NAME.pid
+		#echo $SOCKET
+		if [ -e $SOCKET ]; then
+			rm $SOCKET
+		fi
+	fi
+	echo "$NAME."
+	;;
+  force-reload)
+        echo -n "Force reloading $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        sleep 1
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+        ;;
+  restart)
+        echo "Restarting $DESC: "
+        echo -n "Stopping $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        echo "$NAME."
+	sleep 1
+        echo -n "Starting $DESC: "
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+	;;
+  status)
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
+        ;;
+
+  *)
+	N=/etc/init.d/$NAME
+	echo "Usage: $N {start|stop|force-reload|restart|}" >&2
+	exit 1
+	;;
+esac
+
+exit 0

system/dkimpy-milter.openrc

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="/usr/local/etc/dkimpy-milter/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="/usr/local/bin/dkimpy-milter"
+pidfile="/run/dkimpy-milter/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.openrc.in

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="@CONFDIR@/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="@BINDIR@/dkimpy-milter"
+pidfile="@RUNSTATEDIR@/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.service

@@ -1,11 +1,12 @@
 [Unit]
 Description=DKIMpy Milter
-After=syslog.target network.target
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5) 
+After=network.target
 
 [Service]
 Type=simple
-PIDFile=/var/run/dkimpy-milter/dkimpy-milter.pid
+PIDFile=/run/dkimpy-milter/dkimpy-milter.pid
-ExecStart=/usr/local/bin/dkimpy-milter 
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf -P /run/dkimpy-milter/dkimpy-milter.pid
 
 [Install]
 WantedBy=multi-user.target

system/dkimpy-milter.service.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5) 
+After=network.target
+
+[Service]
+Type=simple
+PIDFile=@RUNSTATEDIR@/dkimpy-milter.pid
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf -P @RUNSTATEDIR@/dkimpy-milter.pid
+
+[Install]
+WantedBy=multi-user.target

system/socket-activation/README.md

@@ -0,0 +1,32 @@
+This directory contains example systemd unit files for running a
+supervised, socket-activated instance of dkimpy-milter.
+
+There are several advantages of using socket activation:
+
+- dkimpy-milter never runs with elevated privileges, they are dropped
+  before any dkimpy-milter code is executed.
+   
+- The socket is opened before dkimpy-milter runs.  This means that
+  clients can connect() to the socket immediately.  So even if there
+  is a delay in dkimpy-milter startup, or in libmilter itself, the
+  connection will not fail.
+  
+- You can set the privileges of a listening Unix-domain socket by an
+  override of ListenGroup= in dkimpy-milter.socket (see
+  systemd.unit(5) for how to override).  This lets you control who has
+  access to the daemon with finer granularity than is available with
+  dkimpy-milter on its own.
+  
+- dkimpy-milter will not consume system resources if it is not used.
+
+- A fully-supervised dkimpy-milter needs no PIDFile, UMask, UserID, or
+  Socket configuation.  This eliminates common race conditions and
+  startup failures, and simplifies the resulting configuration file.
+  
+There is one downside to using socket activation:
+
+- it will only work on systems where libmilter can support connection
+  strings like "fd:3".  This has been supported on Debian and derived
+  systems since sendmail 8.14.4-6 (before Debian Jessie, in early
+  2014), see for example:
+  https://sources.debian.org/src/sendmail/8.15.2-8/debian/patches/socket_activation.patch/

system/socket-activation/dkimpy-milter.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.service.in

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.socket

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=/run/dkimpy-milter/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

system/socket-activation/dkimpy-milter.socket.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=@RUNSTATEDIR@/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

tests/00_minimal.miltertest

@@ -0,0 +1,12 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." complete")
+   end
+end

tests/01_connect.miltertest

@@ -0,0 +1,40 @@
+-- -*- lua -*-
+for _, keytype in ipairs({"ed25519", "rsa"}) do
+   for _, func in ipairs({"signing", "verify"}) do
+      mt.echo("testing "..keytype.." "..func)
+      conn = mt.connect("unix:"..keytype.."."..func..".sock")
+      if conn == nil then
+         error("mt.connect() failed "..keytype.." "..func)
+      end
+      if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+         error("mt.conninfo() failed "..keytype.." "..func)
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.conninfo() unexpected reply "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_ADDHDRS) then
+         print("could add headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not add headers "..keytype.." "..func)
+      end
+
+      if mt.test_action(conn, SMFIF_CHGHDRS) then
+         print("could change headers "..keytype.." "..func)
+      else
+         error("mt.test_action() says could not change headers "..keytype.." "..func)
+      end
+
+-- -- FIXME: this part of the test fails, as apparently the
+-- -- dkimpy-milter claims the right to change the body of a message,
+-- -- even though it shouldn't.  How can we fix the negotiation?
+--      if mt.test_action(conn, SMFIF_CHGBODY) then
+--         error("mt.test_action() says could change body "..keytype.." "..func)
+--      else
+--         print("could not change body "..keytype.." "..func)
+--      end
+
+      mt.disconnect(conn)
+      mt.echo(keytype.." "..func.." test complete")
+   end
+end

tests/02_sign_message.miltertest

@@ -0,0 +1,100 @@
+-- -*- lua -*-
+
+msg = {
+   ['headers'] = {
+      ['From'] = 'Alice <alice@example.net>',
+      ['Message-Id'] = '<dkimpy-milter-test-02@example.net>',
+      ['To'] = 'Bob <bob@example.biz>',
+      ['Date'] = 'Mon, 18 Feb 2019 08:32:50 -0500',
+      ['Subject'] = 'Signing test',
+      ['Content-Type'] = 'text/plain',
+   },
+   ['body'] = "This is a test!\r\n",
+}
+
+-- returns miltertest connection object
+function connect_and_send (sockname, headers, body)
+   conn = mt.connect(sockname)
+   if conn == nil then
+      error "mt.connect() failed"
+   end
+   if mt.conninfo(conn, "localhost", "127.0.0.1") ~= nil then
+      error "mt.conninfo() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.conninfo() unexpected reply"
+   end
+
+   -- mt.macro(conn, SMFIC_MAIL, "i", "simple-message")
+   if mt.mailfrom(conn, "<alicüe@example.net> (Alicþþÿÿe)") ~= nil then
+      error "mt.mailfrom() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.mailfrom() unexpected reply"
+   end
+   -- mt.rcptto() is called implicitly
+
+   -- send headers
+   for key,value in pairs(headers) do
+      if mt.header(conn, key, value) ~= nil then
+         error("mt.header(" .. key .. ") failed")
+      end
+      if mt.getreply(conn) ~= SMFIR_CONTINUE then
+         error("mt.header(" .. key .. ") unexpected reply")
+      end
+   end
+   -- send EOH
+   if mt.eoh(conn) ~= nil then
+      error "mt.eoh() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.eoh() unexpected reply"
+   end
+
+   -- send body
+   if mt.bodystring(conn, body) ~= nil then
+      error "mt.bodystring() failed"
+   end
+   if mt.getreply(conn) ~= SMFIR_CONTINUE then
+      error "mt.bodystring() unexpected reply"
+   end
+   -- end of message; let the filter react
+   if mt.eom(conn) ~= nil then
+      error "mt.eom() failed"
+   end
+   reply = mt.getreply(conn)
+   if reply ~= SMFIR_CONTINUE then
+      error ("mt.eom() unexpected reply: " .. reply)
+   end
+   return conn
+end
+
+for _, keytype in ipairs({"ed25519", "rsa", "ed25519.stable", "rsa.stable", "ed25519.table", "rsa.table"}) do
+   mt.echo("testing "..keytype)
+   signing = connect_and_send("unix:"..keytype..".signing.sock", msg.headers, msg.body)
+   -- verify that a test header field got added
+   if not mt.eom_check(signing, MT_HDRINSERT) then
+      error "no header added by signer"
+   end
+
+   signature = mt.getheader(signing, "DKIM-Signature", 0)
+
+   mt.disconnect(signing)
+
+   mt.echo("DKIM-Signature: " .. signature)
+
+   msg.headers['DKIM-Signature'] = signature
+
+   verify = connect_and_send("unix:"..keytype..".verify.sock", msg.headers, msg.body)
+
+   if not mt.eom_check(verify, MT_HDRINSERT) then
+      error "no header added in verify"
+   end
+
+   authres = mt.getheader(verify, "Authentication-Results", 0)
+   mt.echo("Authentication-Results: "..authres)
+
+   mt.disconnect(verify)
+
+   mt.echo(keytype.." complete")
+end

tests/dkimpy-milter

@@ -0,0 +1,2 @@
+#!/bin/sh
+python3 -m dkimpy_milter "$@"

tests/runtests

@@ -0,0 +1,143 @@
+#!/bin/bash
+
+set -e
+WORKDIR=$(mktemp -d)
+TESTDIR=$(realpath "$(dirname "$0")")
+DKIMPY_MILTER=${DKIMPY_MILTER:-"$TESTDIR/dkimpy-milter"}
+KEY_TYPES=(ed25519 rsa)
+
+cd "$WORKDIR"
+
+printf "Testing %s from directory %s\n" "$DKIMPY_MILTER" "$WORKDIR"
+
+for keytype in "${KEY_TYPES[@]}"; do
+    dknewkey --ktype "$keytype" "testkey.$keytype"
+    if [ "$keytype" = ed25519 ]; then
+        keyfile=KeyFileEd25519
+        selector=SelectorEd25519
+    elif [ "$keytype" = rsa ]; then
+        keyfile=KeyFile
+        selector=Selector
+    fi
+    if [ "$keytype" = ed25519 ]; then
+        keytable=KeyTableEd25519
+        signingtable=SigningTable
+        selector=SelectorEd25519
+    elif [ "$keytype" = rsa ]; then
+        keytable=KeyTable
+        signingtable=SigningTable
+        selector=Selector
+    fi
+    cat > "$keytype.signing.conf" <<EOF
+Domain example.net
+$keyfile testkey.$keytype.key
+$selector testkey
+Socket unix:$keytype.signing.sock
+PidFile $keytype.signing.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.verify.conf" <<EOF
+Socket unix:$keytype.verify.sock
+PidFile $keytype.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+MinimumKeyBits 2048
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.conf" <<EOF
+$keyfile testkey.$keytype.key
+$selector testkey
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.stable.signing.sock
+PidFile $keytype.stable.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.verify.conf" <<EOF
+Socket unix:$keytype.stable.verify.sock
+PidFile $keytype.stable.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.table.conf" <<EOF
+$keytable $WORKDIR/$keytype-table
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.table.signing.sock
+PidFile $keytype.table.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+cat > "$keytype.table.verify.conf" <<EOF
+Socket unix:$keytype.table.verify.sock
+PidFile $keytype.table.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat >  "$keytype-table" <<EOF
+preskey example.org:testkey:$WORKDIR/testkey.$keytype.key
+orgkey  example.org:testkey:$WORKDIR/testkey.$keytype.key
+netkey  example.net:testkey:$WORKDIR/testkey.$keytype.key
+EOF
+
+    cat > "signing-table" <<EOF
+president@example.org	@special.example.org:preskey
+*@example.org		orgkey
+*@example.net		netkey
+EOF
+done
+
+cleanup() {
+    echo cleaning up jobs:
+    jobs
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify stable stable.verify table table.verify; do
+            if [ -s "$keytype.$func.pid" ] && kill -0 "$(cat "$keytype.$func.pid")"; then
+                kill "$(cat $keytype.$func.pid)"
+            fi
+        done
+    done
+    wait
+    for keytype in "${KEY_TYPES[@]}"; do
+        for func in signing verify stable stable.verify table table.verify; do
+            errdata="$keytype.$func.stderr"
+            if [ -s "$errdata" ]; then
+                printf -- "-> %s:\n" "$errdata"
+                cat "$errdata"
+                printf -- "-> end %s\n" "$errdata"
+            fi
+        done
+    done
+    rm -rf "$WORKDIR"
+}
+
+for keytype in "${KEY_TYPES[@]}"; do
+    for func in signing verify stable stable.verify table table.verify; do
+        PYTHONPATH="$(dirname "$TESTDIR")" "$DKIMPY_MILTER" "$keytype.$func.conf" 2>"$keytype.$func.stderr" &
+    done
+done
+trap cleanup EXIT
+
+# ugly ugly (how are we supposed to know that the milters are all ready?):
+sleep 2
+
+# uses miltertest from opendkim:
+for x in ${TESTS:-"$TESTDIR"/*.miltertest}; do
+    if ! [ -e "$x" ]; then
+        if [ -e "$TESTDIR/$x" ]; then
+            x="$TESTDIR/$x"
+        fi
+    fi
+    printf -- "-> running %s...\n" "$x"
+    miltertest -s "$x"
+done
+
+rm -rf "$(dirname $TESTDIR)/dkimpy_milter/__pycache__"