A-label

(LP: #1901445)

CHANGES

@@ -1,25 +1,66 @@
-1.1.4 2019-11-22
+1.2.3 2023-02-26
- - Make error logging more explicit to aid debugging
+ - Improve support for non-ASCII email messages.  Anything UTF-8 should work
- - Delete own_socketfile to resolve race condition where the permissions
+   (including correct signing/verification).  For messages that contain header
-   change fails on a Unix socket because it hasn't been created yet (libmilter
+   fields with non-ASCII or UTF-8 content, signatures are likely fail
-   will do this correctly on its own based on umask, the milter doesn't need
+   verification, but the milter should continue to run.  (Thanks to Casper
-   to do it) (LP: #1849712)
+   Bruun for help with this)
+ - Set minimum pymilter and dkimpy versions in setup.py to those that will
+   work reliably with non-ASCII content.
+ - Fixed support for percent in KeyTable - Thanks to Mika Tiainen
+ - Fix formatting for MinimumKeyBits in dkimpy-milter.conf(5)
+   (Closes: #995335)
+ - Reset the i= signature identity in get_identities_sign() (Closes: #981157)
+ - Improve documentation of inter-relationship between Mode, InternalHosts,
+   MacroList, and MacroListVerify options in dkimpy-milter.conf.5 (Closes:
+   #969215)
+ - Fix subdomain signing with top-level organizational domain (LP: #1999434)
+   - Thanks to Matthias Hunstock for the report and the fix
+ - Fix comma separated list processing in dkimpy_milter/config.py
+   (LP: #1901445)
 
-1.1.3 2019-10-06
+1.2.2 2020-08-09
- - Fix sysv init so it works (LP: #1839487)
+ - Improve README.md formating for markdown display on pypi
+ - Improve documentation in dkimpy-milter.conf (5) and README.md for signing
+   for multiple domains (Thanks to Stefano Rivera)
+ - Minimal fix for dnspython 2.0.0 compatibility (still works with 1.16.0)
 
-1.1.2 2019-09-23
+1.2.1 2020-01-04
- - Fix variable initialization so mailformed mails missing body From do not
+ - Fix expand option not to fail if files are missing since socket activation
-   cause a traceback (LP: #1844161)
+   service files are not shipped in the sdist
- - Catch more ascii encoding errors to improve resilience against bad data
+ - Correct dkimpy-milter.conf file install location to match expand locations
-   (LP: #1844189)
 
-1.1.1 2019-09-06
+1.2.0 2020-01-03
+ - Add support for SigningTable, KeyTable, and KeyTableEd25519 (LP: #1797397)
+ - Add support for specifying MinimumKeyBits for RSA signatures
+ - Add support for SignHeaders feature, thanks to Ralph Seichter for the patch
+ - Add support for specifying DNSTimeout (bumps required dkimpy version to 1.0)
+ - Add information on message content conversion to README
+ - Add new expand option to setup.py so various file system locations can be
+   specified at build/install time rather than being hard coded
+ - Install openrc init file for Gentoo and other openrc users
+ - Add support for passing PID file name on command line to make it easier to
+   keep system init and daemon configuration in sync
+ - Add support for storing DKIM failed mails in a specified
+   DiagnosticDirectory
  - Fix startup logging so it provides information at a useful time
  - Fix verify processing so missing (optional) i= tag doesn't cause the milter
    to fail  (LP: #1842250)
  - Fix message extraction so that signing in the same pass through the milter
    as verifying works correctly
+ - Add debug logging for content type to assist troubleshooting MIME
+   conversion issues
+ - Fix variable initialization so mailformed mails missing body From do not
+   cause a traceback (LP: #1844161)
+ - Catch more ascii encoding errors to improve resilience against bad data
+   (LP: #1844189)
+ - Fix sysv init so it works (LP: #1839487)
+ - Make error logging more explicit to aid debugging
+ - Remove SigningTableEd25519 from documentation - it was never implemented
+   and a per algorithm signing table turns out not to be needed
+ - Delete own_socketfile to resolve race condition where the permissions
+   change fails on a Unix socket because it hasn't been created yet (libmilter
+   will do this correctly on its own based on umask, the milter doesn't need
+   to do it) (LP: #1849712)
 
 1.1.0 2019-04-12
  - Add SubDomains option to enable signing for sub-domains (LP: #1811535)

Cheatsheet.txt

@@ -0,0 +1,25 @@
+Welcome to Cheatsheet.txt, licensed under CC-0. No attribution required.
+But it is writen by Diskette (diskette@dailitation.xyz)
+
+Information regarding the [] flags are in dkimpy repository.
+
+[__init__.py]
+Initialization file,
+
+class dkimMilter, a milter for dkim
+
+What are those
+self.fp localpart
+self.fdomain domain part
+self.iequals i still have no idea
+
+def header
+
+
+define check_dkim, I assume that this checks dkim, how?
+
+def sign_dkim
+d = dkim.DKIM(txt)
+
+[dnsplyug.py]
+File for interfacing with DNS

README

@@ -1,242 +0,0 @@
-OVERVIEW
-========
-
-This is a DKIM signing and verification milter.  It has been tested with both
-Postfix and Sendmail.
-
-The configuration file is designed to be compatible with OpenDKIM, but only
-a subset of OpenDKIM options are supported.  If an unsupported option is
-specified, an error will be raised.
-
-
-INSTALLATION
-===========
-
-This package includes a default configuration file and man pages.  For those
-to be installed when installing using setup.py, the following incantation is
-required because setuptools developers decided not being able to do this by
-default is a feature:
-
-python3 setup.py install --single-version-externally-managed --record=/dev/null
-
-For users of Debian Stable (Debian 9, Codename Squeeze), all dependencies are
-available in either the main or backports repositories:
-
-[sudo] apt install python3-milter python3-nacl python3-dnspython
-[sudo] apt install -t stretch-backports python3-authres python3-dkim
-
-The preferred method of installation is from PyPi using pip (if distribution
-packages are not available):
-
-[sudo] pip install dkimpy_milter
-
-Using pip will cause required packages to be installed via easy_install if they
-have not been previously installed.  Because pymilter and PyNaCl are compiled
-Python extensions, the system will need appropriate development packages and
-an C compiler.  Alternately, install these dependencies from distribution/OS
-packages and then pip install dkimpy_milter.
-
-The milter will work with either py3dns (DNS) or dnspython (dns), preferring
-dnspython if both are available.  The dkimpy DKIM module also works with
-either.
-
-
-SETUP
-====
-
-SIGNING KEYS
-============
-
-In order to create DKIM signatures, a private key must be available.  Signing
-keys should be protected (owned by root:root with permissions 600 in a
-directory that is not world readable).  Different keys are required for RSA
-and (if used) Ed25519.
-
-RSA
-===
-
-Both public and private keys for RSA have standard formats and there are many
-tools available to create them.  Keys must (RFC 8302) have a minimum size of
-1024 bits and should have a size of at least 2048 bits.  The dknewkey script
-that is provided with dkimpy is one such tool:
-
-dknewkey exampleprivkey
-
-will produce both the private key file (.key suffix) and a file with the DKIM
-public key record to be published DNS (.dns suffix).  RSA is the default key
-type.  2048 bits is the default key size.
-
-ED25519
-=======
-
-There is no standardized non-binary representation for Ed25519 private keys,
-so in order to generate Ed25519 keys for dkimpy-milter, dkimpy specific tools
-must be used to be compatible.  The same dknewkey script support Ed25519:
-
-dknewkey --ktype ed25519 anothernewkey
-
-will provide both the private key file (.key suffix) and a file with the DKIM
-public key record to be published DNS (.dns suffix).  Ed25519 keys do not have
-variable bit lengths.
-
-MTA INTEGRATION
-==============
-
-Both a systemd unit file and a sysv init file are provided.  Both make
-assumptions about defaults being used, e.g. if a non-standard pidfile name is
-used, they will need to be updated.  The sysv init file uses start-stop-deamon
-from Debian.  It is not portable to systems without that available.
-
-The dkimpy-milter drops priviledges after setup to the user/group specified in
-UserID.  During initial setup, this system user needs to be manually created.
-As an example, using the default dkimpy-user on Debian, the command would be:
-
-[sudo] adduser --system --no-create-home --quiet --disabled-password \
-               --disabled-login --shell /bin/false --group \
-               --home /run/dkimpy-milter dkimpy-milter
-
-Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
-missing, the milter will create it on startup.
-
-To start dkimpy-milter with systemd for the first time, you will need to take
-the following steps:
-
-[sudo] systemctl daemon-reload
-[sudo] systemctl enable dkimpy-milter
-[sudo] systemctl start dkimpy-milter
-[sudo] systemctl status dkimpy-milter (to verify it started correctly)
-
-As with all milters, dkimpy-milter needs to be integrated with your MTA of
-choice (Sendmail or Postfix).
-
-SENDMAIL
-========
-
-Configuration is very similar to opendkim, but needs some adjustment for
-dkimpy-milter. Here's an example configuration line to include in your
-sendmail.mc:
-
-INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/run/dkimpy-milter/dkimpy-milter.sock')dnl
-
-Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
-and a restart of sendmail.  Note that S= needs to match the value of Socket in
-the dkimpy-milter configuration file.
-
-Milter support should be present by default in most versions of sendmail
-these days, but if not included in your Sendmail build, see:
-http://www.elandsys.com/resources/sendmail/milter.html
-
-ISSUES USING SENDMAIL TO SIGN AND VERIFY
-========================================
-
-When using the sendmail MTA in both signing and verifying mode, there are
-a few issues of which to be aware that might cause operational problems
-and deserve consideration.
-
-(a) When the MTA will be used for relaying emails, e.g. delivering to other
-    hosts using the aliases mechanism, it is important not to break
-    signatures inserted by the original sender.  This is particularly sensitive
-    particular when the sending domain has published a "reject" DMARC policy.
-
-    By default, sendmail quotes to address header fields when there are no
-    quotes and the display part of the address contains a period or an
-    apostrophe.  However, opendkim only sees the raw, unmodified form of
-    the header field, and so the content that gets verified and what gets
-    signed will not be the same, guaranteeing the attached signature is not
-    valid.
-
-    To direct sendmail not to modify the headers, add this to your sendmail.mc:
-
-    	conf(`confMUST_QUOTE_CHARS', `')
-
-(b) As stated in sendmail's KNOWNBUGS file, sendmail truncates header field
-    values longer than 256 characters, which could mean truncating the domain
-    of a long From: header field value and invalidating the signature.
-    You may wish to consider increasing MAXNAME in sendmail/conf.h to mitigate
-    changing the messages and invalidating their signatures.  This change
-    requires recompiling sendmail.
-
-(c) Similar to (a) above, sendmail may wrap very long single-line recipient
-    fields for presentation purposes; for example:
-
-    To: very long name <a@example.org>,anotherloo...ong name b <b@example.org>
-
-    ...might be rewritten as:
-
-    To: very long name <a@example.org>,
-    	anotherloo...ong name b <b@example.org>
-
-    This rewrite is also done after opendkim has seen the message, meaning
-    the signature opendkim attaches to the message does not match the
-    content it signed.  There is not a known configuration change to
-    mitigate this mutation.
-
-    The only known mechanism for dealing with this is to have distinct
-    instances of opendkim do the verifying (inbound) and signing (outbound)
-    so that the version that arrives at the signing instance is already
-    in the rewritten form, guaranteeing the input and output are the same
-    and thus the signature matches the payload.
-
-POSTFIX
-=======
-
-Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
-README_FILES/MILTER_README).  Here's an example master.cf excerpt that talks
-to two dkimpy-milter instances, one configured for signing and one configured
-for verification:
-
-smtp       inet  n       -       -       -      -       smtpd
-    ...
-    -o smtpd_milters=inet:localhost:8892
-    ...
-
-submission inet  n       -       -       -      -       smtpd
-    ...
-    -o smtpd_milters=inet:localhost:8891
-    ...
-
-These need to match the Socket value for each dkimpy-milter instance.
-
-Care is required to segregate outbound mail to be signed and inbound mail to
-be verified.  The above example uses two instances of dkimpy-milter to do
-this.  There are many possible ways.  Here is another example using milter
-macros to keep the mail streams segregated:
-
-Postfix master.cf:
-
-smtp       inet  n       -       -       -       -       smtpd
-    ...
-    -o smtpd_milters=inet:localhost:8891
-    -o milter_macro_daemon_name=VERIFYING
-    ...
-
-submission inet n       -       -       -       -       smtpd
-    -o syslog_name=postfix/submission
-    -o smtpd_tls_security_level=encrypt
-    -o smtpd_sasl_auth_enable=yes
-    ...
-    -o milter_macro_daemon_name=ORIGINATING
-    -o smtpd_milters=inet:localhost:8891
-    ...
-
-Dkimpy-milter.conf:
-
-...
-Mode			sv
-MacroList		dameon_name|ORIGINATING
-MacroListVerify		daemon_name|VERIFYING
-...
-
-
-NOTES
-=====
-
-The python DKIM library, dkimpy, requires the entire message being signed or
-verified to be in memory, so dkimpy-milter does not write messages out to a
-temp file.  This may impact performance on low-memory systems.
-
-DKIM with Ed25519 signatures are described in RFC 8463.  Version 1.0.0 and
-later support Ed25519 signing and verification.  RFC 8301 removed rsa-sha1
-from DKIM.  dkimpy-milter does not sign with rsa-sha1, but still considers
-rsa-sha1 signatures as valid for verification because they are still in
-common use and are not known to be cryptographically broken.

README.md

@@ -0,0 +1,351 @@
+An SMTPUTF8-approved version of dkimpy-milter
+
+Please do note that there might be some mistakes along the way... No warranty is provided!
+
+This implements support for internationalized email address (RFC 8616)
+
+Cheatsheet.txt is a file for my (diskette@dailitation.xyz) personal note taking.
+
+# OVERVIEW
+
+This is a DKIM signing and verification milter.  It has been tested with both
+Postfix and Sendmail.
+
+The configuration file is designed to be compatible with OpenDKIM, but only
+a subset of OpenDKIM options are supported.  If an unsupported option is
+specified, an error will be raised.
+
+
+# INSTALLATION
+
+This package includes a default configuration file and man pages.  For those
+to be installed when installing using setup.py, the following incantation is
+required because setuptools developers decided not being able to do this by
+default is a feature:
+
+[sudo] python3 setup.py install --single-version-externally-managed --record=/dev/null
+
+For users of Debian Stable (Debian 9, Codename Squeeze), all dependencies are
+available in either the main or backports repositories:
+
+[sudo] apt install python3-milter python3-nacl python3-dnspython
+[sudo] apt install -t stretch-backports python3-authres python3-dkim
+
+It is also available in the Debian package archive:
+
+[sudo] apt install dkimpy-milter [Debian 10 or later]
+[sudo] apt install -t stretch-backports dkimpy-milter [Debian 9]
+
+When installing using the Debian package, all dependencies are automatically
+installed.
+
+The preferred method of installation is from PyPi using pip (if distribution
+packages are not available):
+
+[sudo] pip install dkimpy_milter
+
+Using pip will cause required packages to be installed via easy_install if they
+have not been previously installed.  Because pymilter and PyNaCl are compiled
+Python extensions, the system will need appropriate development packages and
+an C compiler.  Alternately, install these dependencies from distribution/OS
+packages and then pip install dkimpy_milter.
+
+The milter will work with either py3dns (DNS) or dnspython (dns), preferring
+dnspython if both are available.  The dkimpy DKIM module also works with
+either.
+
+## NON-STANDARD INSTALLATION PATHS
+
+The package includes a custom setup command called expand.  It allows various
+file locations in init scripts, man pages, and config files to be over-ridden
+at install time.
+
+
+    expand: Expand @@ variables in input files, simlar to make macros.
+    user_options:
+      --sysconfigdir=, e: Specify system configuration directory.
+      --sbindir=, s: Specify system binary directory [not used].
+      --bindir=, b: Specify binary directory.
+      --rundir=,r: Specify run state directory.
+
+As an example, to change the run directory to /var/run, one would do:
+
+    python3 setup.py expand --rundir=/var/run
+    [sudo] python3 setup.py install --single-version-externally-managed \
+                                --record=/dev/null
+
+or in a single step (the order matters):
+
+    [sudo] python3 setup.py expand --rundir=/var/run install \
+                               --single-version-externally-managed \
+                               --record=/dev/null
+
+# SETUP
+
+## SIGNING KEYS
+
+In order to create DKIM signatures, a private key must be available.  Signing
+keys should be protected (owned by root:root with permissions 600 in a
+directory that is not world readable).  Different keys are required for RSA
+and (if used) Ed25519.
+
+### RSA
+
+Both public and private keys for RSA have standard formats and there are many
+tools available to create them.  Keys must (RFC 8302) have a minimum size of
+1024 bits and should have a size of at least 2048 bits.  The dknewkey script
+that is provided with dkimpy is one such tool:
+
+    dknewkey exampleprivkey
+
+will produce both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  RSA is the default key
+type.  2048 bits is the default key size.
+
+### ED25519
+
+There is no standardized non-binary representation for Ed25519 private keys,
+so in order to generate Ed25519 keys for dkimpy-milter, dkimpy specific tools
+must be used to be compatible.  The same dknewkey script support Ed25519:
+
+    dknewkey --ktype ed25519 anothernewkey
+
+will provide both the private key file (.key suffix) and a file with the DKIM
+public key record to be published DNS (.dns suffix).  Ed25519 keys do not have
+variable bit lengths.
+
+### COMPLEX SIGNING CONFIGURATIONS
+
+The KeyTable, KeyTableEd25519, and SigningTable are used to define signing
+instructions to the filter where use of Domain, Selector and KeyFile together
+are insufficient.
+
+First, select the type of database you will use for each.  They need not
+be the same.  The "DATA SETS" portion of the dkimpy-milter(8) man page
+describes the possibilities and how they are formatted.  Then, construct those
+databases.
+
+Let's suppose you want to sign for two domains, example.com and example.net.
+Within example.com, you want to sign for user "president" differently than
+everyone else.  Let's say further that you want to use a flat text file.
+
+You've generated private key files for each of these and stored them
+in the directory /usr/local/etc/dkim/keys as files "president", "excom" and
+"exnet", with the obvious intents.  You want to use selectors "foo", "bar"
+and "baz" for those, respectively.  The signing domains match the senders
+(i.e. the signatures for example.com's stuff will be held by example.com,
+and example.net likewise).
+
+First, write the KeyTable.  This is a list of the keys you intend to use,
+and you just assign arbitrary names to them.  So as a flat file, the KeyTable
+for the above might look like this:
+
+	preskey	example.com:foo:/usr/local/etc/dkim/keys/president
+	comkey	example.com:bar:/usr/local/etc/dkim/keys/excom
+	netkey	example.net:baz:/usr/local/etc/dkim/keys/exnet
+
+If also signing with ed25519, specify a KeyTableEd25519, with the same
+names, pointing to the keys needed for ed25519.  Both KeyTable and
+KeyTableEd25519 are evaluated if there is a SigningTable (see below).
+
+Per the documentation, multi-field data sets that are made of flat files have
+the fields separated by colons, but the key and value(s) are separated by
+whitespace.
+
+So now we've named each key file, and specified with which selector and domain
+each will be used, and then given each of those groupings a name.  This
+is your KeyTable.  Let's say you put it in /usr/local/etc/dkim/keytable.
+
+Next, write the SigningTable.  This maps senders (by default, taken from the
+From: header field of a message passing through the filter) to which keys
+will be used to sign their mail.  Wildcards are allowed.  So to do what was
+described above, we write it as follows:
+
+	president@example.com	preskey
+	*@example.com		comkey
+	*@example.net		netkey
+
+Since we want to use wildcards, we can't actually use a regular flat file.
+Wildcards require a regular expression file, or "refile".  The above is
+valid format for one of those.  Let's say you put this in
+/usr/local/etc/dkim/signingtable.
+
+Finally, tell the filter that it should use these files by adding this to
+your configuration file:
+
+	KeyTable	/usr/local/etc/dkim/keytable
+	SigningTable	refile:/usr/local/etc/dkim/signingtable
+
+You could put "file:" in front of the filename for the KeyTable just to be
+precise, but "file:" is assumed if the value starts with a "/".
+
+Note: Unlike opendkim, dkimpy-milter will check for "\*" in the signing table
+regardless of if refile is specified or not.  Use of refile is supported for
+compatibility with configurations initially developed for use with opendkim.
+
+## MTA INTEGRATION
+
+Both a systemd unit file and a sysv init file are provided.  Both make
+assumptions about defaults being used, e.g. if a non-standard pidfile name is
+used, they will need to be updated.  The sysv init file uses start-stop-deamon
+from Debian.  It is not portable to systems without that available.
+
+The dkimpy-milter drops priviledges after setup to the user/group specified in
+UserID.  During initial setup, this system user needs to be manually created.
+As an example, using the default dkimpy-user on Debian, the command would be:
+
+    [sudo] adduser --system --no-create-home --quiet --disabled-password \
+               --disabled-login --shell /bin/false --group \
+               --home /run/dkimpy-milter dkimpy-milter
+
+Since /var/run or /run is sometimes on a tempfs, if the PID file directory is
+missing, the milter will create it on startup.
+
+To start dkimpy-milter with systemd for the first time, you will need to take
+the following steps:
+
+    [sudo] systemctl daemon-reload
+    [sudo] systemctl enable dkimpy-milter
+    [sudo] systemctl start dkimpy-milter
+    [sudo] systemctl status dkimpy-milter (to verify it started correctly)
+
+As with all milters, dkimpy-milter needs to be integrated with your MTA of
+choice (Sendmail or Postfix).  When integrating with your MTA, the risk of
+signature invalidation due to content conversion of the message body needs to
+be considered.  See RFC 6376, Section 5.3 for discussion of this issue.  As a
+practical matter, when signing, configure the milter to follow all others that
+might modify the message body.  When verifying, configure the milter before
+other processes that might modify the message body.
+
+### SENDMAIL
+
+Configuration is very similar to opendkim, but needs some adjustment for
+dkimpy-milter. Here's an example configuration line to include in your
+sendmail.mc:
+
+    INPUT_MAIL_FILTER(`dkimpy-milter', `S=local:/run/dkimpy-milter/dkimpy-milter.sock')dnl
+
+Changing the sendmail.mc file requires a Make (to compile it into sendmail.cf)
+and a restart of sendmail.  Note that S= needs to match the value of Socket in
+the dkimpy-milter configuration file.
+
+Milter support should be present by default in most versions of sendmail
+these days, but if not included in your Sendmail build, see:
+http://www.elandsys.com/resources/sendmail/milter.html
+
+#### ISSUES USING SENDMAIL TO SIGN AND VERIFY
+
+When using the sendmail MTA in both signing and verifying mode, there are
+a few issues of which to be aware that might cause operational problems
+and deserve consideration.
+
+(a) When the MTA will be used for relaying emails, e.g. delivering to other
+    hosts using the aliases mechanism, it is important not to break
+    signatures inserted by the original sender.  This is particularly sensitive
+    particular when the sending domain has published a "reject" DMARC policy.
+
+    By default, sendmail quotes to address header fields when there are no
+    quotes and the display part of the address contains a period or an
+    apostrophe.  However, dkimpy-milter only sees the raw, unmodified form of
+    the header field, and so the content that gets verified and what gets
+    signed will not be the same, guaranteeing the attached signature is not
+    valid.
+
+    To direct sendmail not to modify the headers, add this to your sendmail.mc:
+
+    	conf(`confMUST_QUOTE_CHARS', `')
+
+(b) As stated in sendmail's KNOWNBUGS file, sendmail truncates header field
+    values longer than 256 characters, which could mean truncating the domain
+    of a long From: header field value and invalidating the signature.
+    You may wish to consider increasing MAXNAME in sendmail/conf.h to mitigate
+    changing the messages and invalidating their signatures.  This change
+    requires recompiling sendmail.
+
+(c) Similar to (a) above, sendmail may wrap very long single-line recipient
+    fields for presentation purposes; for example:
+
+    To: very long name <a@example.org>,anotherloo...ong name b <b@example.org>
+
+    ...might be rewritten as:
+
+    To: very long name <a@example.org>,
+    	anotherloo...ong name b <b@example.org>
+
+    This rewrite is also done after dkimpy-milter has seen the message,
+    meaning the signature dkimpy-milter attaches to the message does not match
+    the content it signed.  There is not a known configuration change to
+    mitigate this mutation.
+
+    The only known mechanism for dealing with this is to have distinct
+    instances of dkimpy-milter do the verifying (inbound) and signing
+    (outbound) so that the version that arrives at the signing instance is
+    already in the rewritten form, guaranteeing the input and output are the
+    same and thus the signature matches the payload.
+
+### POSTFIX
+
+Integration of dkimpy-milter into Postfix is like any milter (See Postfix's
+README_FILES/MILTER_README).  Here's an example master.cf excerpt that talks
+to two dkimpy-milter instances, one configured for signing and one configured
+for verification:
+
+    smtp       inet  n       -       -       -      -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8892
+        ...
+
+    submission inet  n       -       -       -      -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8891
+        ...
+
+These need to match the Socket value for each dkimpy-milter instance.
+
+Care is required to segregate outbound mail to be signed and inbound mail to
+be verified.  The above example uses two instances of dkimpy-milter to do
+this.  There are many possible ways.  Here is another example using milter
+macros to keep the mail streams segregated:
+
+Postfix master.cf:
+
+    smtp       inet  n       -       -       -       -       smtpd
+        ...
+        -o smtpd_milters=inet:localhost:8891
+        -o milter_macro_daemon_name=VERIFYING
+        ...
+
+    submission inet n       -       -       -       -       smtpd
+        -o syslog_name=postfix/submission
+        -o smtpd_tls_security_level=encrypt
+        -o smtpd_sasl_auth_enable=yes
+        ...
+        -o milter_macro_daemon_name=ORIGINATING
+        -o smtpd_milters=inet:localhost:8891
+        ...
+
+Dkimpy-milter.conf:
+
+    ...
+    Mode			sv
+    MacroList		daemon_name|ORIGINATING
+    MacroListVerify		daemon_name|VERIFYING
+    ...
+
+
+# NOTES
+
+The python DKIM library, dkimpy, requires the entire message being signed or
+verified to be in memory, so dkimpy-milter does not write messages out to a
+temp file.  This may impact performance on low-memory systems.
+
+DKIM with Ed25519 signatures are described in RFC 8463.  Version 1.0.0 and
+later support Ed25519 signing and verification.  RFC 8301 removed rsa-sha1
+from DKIM.  dkimpy-milter does not sign with rsa-sha1, but still considers
+rsa-sha1 signatures as valid for verification because they are still in
+common use and are not known to be cryptographically broken.
+
+Support for non-ASCII email messages:  Anything UTF-8 should work (including
+correct signing/verification).  For messages that contain header fields with
+non-ASCII or UTF-8 content, signatures are likely fail verification, but the
+milter should continue to run.  RFC 8616 is not supported.

TODO

@@ -49,21 +49,25 @@ Port to Python 3  implemented	verified
 Subdomain support implemented	verified
 Test suite	  implemented	verified
 
+1.2.0
+DNSTimeout (dkimpy 1.0)  implemented  verified by inspection
+KeyTable            implemented  verified
+KeytableEd25519     implemented  verified
+MinimumKeyBits      implemented  verified
+SignHeaders	    implemented  verified by inspection
+SigningTable        implemented  verified
+TemporaryDirectory  implemented  verified by inspection
+
 Planned dataset type support (if needed):
 mdb:
 
 Considered for near-term feature release
 
-KeyTable
-KeytableEd25519
-SigningTable
-SigningTableEd25519
 AlwaysAddARHeader
 ChangeRootDirectory
 ClockDrift (requires dkimpy change)
-DNSTimeout (requires dkimpy change)
 MilterDebug
-MinimumKeyBits
+OmitHeaders
 OversignHeaders (may require dkimpy changes)
 PeerList
 SignatureAlgorithm
@@ -85,7 +89,6 @@ MaximumSignaturesToVerify
 MultipleSignatures
 MustBeSigned
 NoHeaderB 
-OmitHeaders
 On-BadSignature
 On-Default
 On-DNSError
@@ -100,11 +103,8 @@ RequiredHeaders
 RequireSafeKeys
 SignatureAlgorithm
 SignatureTTL
-SignHeaders
 SoftwareHeader
 StrictHeaders
-SubDomains
-TemporaryDirectory
 TestDNSData
 TestPublicKeys
 

dkimpy_milter/__init__.py

@@ -31,15 +31,16 @@ import tempfile
 import io
 import re
 import codecs
+import idna
 from Milter.utils import parse_addr, parseaddr
 import dkimpy_milter.config as config
 from dkimpy_milter.util import drop_privileges
 from dkimpy_milter.util import setExceptHook
 from dkimpy_milter.util import write_pid
-from dkimpy_milter.util import read_keyfile
+from dkimpy_milter.util import get_keys
 from dkimpy_milter.util import fold
 
-__version__ = "1.0.1"
+__version__ = "1.2.3"
 FWS = re.compile(r'\r?\n[ \t]+')
 
 
@@ -51,10 +52,9 @@ class dkimMilter(Milter.Base):
         self.id = Milter.uniqueID()
         # we don't want config used to change during a connection
         self.conf = milterconfig
-        self.privatersa = privateRSA
-        self.privateed25519 = privateEd25519
         self.fp = None
         self.fdomain = ''
+        self.iequals = None
 
     @Milter.noreply
     def connect(self, hostname, unused, hostaddr):
@@ -66,19 +66,19 @@ class dkimMilter(Milter.Base):
         if self.receiver is not None:
             self.receiver = self.receiver.strip()
         try:
-            self.AuthservID = milterconfig['AuthservID']
+            self.AuthservID = self.conf['AuthservID']
         except:
             self.AuthservID = self.receiver
         if hostaddr and len(hostaddr) > 0:
             ipaddr = hostaddr[0]
-            if milterconfig['IntHosts']:
+            if self.conf['IntHosts']:
-                if milterconfig['IntHosts'].match(ipaddr):
+                if self.conf['IntHosts'].match(ipaddr):
                     self.internal_connection = True
         else:
             ipaddr = ''
         self.connectip = ipaddr
-        if milterconfig.get('MacroList') and not self.internal_connection:
+        if self.conf.get('MacroList') and not self.internal_connection:
-            macrolist = milterconfig.get('MacroList')
+            macrolist = self.conf.get('MacroList')
             for macro in macrolist:
                 macroname = macro.split('|')[0]
                 macroname = '{' + macroname + '}'
@@ -86,8 +86,8 @@ class dkimMilter(Milter.Base):
                 if ((len(macro.split('|')) == 1 and macroresult) or macroresult
                         in macro.split('|')[1:]):
                     self.internal_connection = True
-        if milterconfig.get('MacroListVerify'):
+        if self.conf.get('MacroListVerify'):
-            macrolist = milterconfig.get('MacroListVerify')
+            macrolist = self.conf.get('MacroListVerify')
             for macro in macrolist:
                 macroname = macro.split('|')[0]
                 macroname = '{' + macroname + '}'
@@ -99,18 +99,31 @@ class dkimMilter(Milter.Base):
             connecttype = 'INTERNAL'
         else:
             connecttype = 'EXTERNAL'
-        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 1:
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 1:
             syslog.syslog("connect from {0} at {1} {2}"
                           .format(hostname, hostaddr, connecttype))
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3:
+            syslog.syslog("internal_conn: {0}, external_conn: {1}"
+                          .format(self.internal_connection, self.external_connection))
+
         return Milter.CONTINUE
 
     # multiple messages can be received on a single connection
     # envfrom (MAIL FROM in the SMTP protocol) seems to mark the start
     # of each message.
     @Milter.noreply
-    def envfrom(self, f, *str):
+    def envfrom(self, f, *moredata):
-        if milterconfig.get('Syslog') and milterconfig.get('debugLevel') >= 2:
+        try:
-            syslog.syslog("mail from: {0} {1}".format(f, str))
+            f = str(codecs.encode(f, 'UTF-8', 'replace'), 'UTF-8', 'ignore')
+        except TypeError:
+            f = codecs.encode(f, 'UTF-8', 'replace').decode()
+        try:
+            moredata = str(codecs.encode(str(moredata), 'UTF-8', 'replace'), 'UTF-8', 'ignore')
+        except TypeError:
+            moredata = codecs.encode(str(moredata), 'UTF-8', 'replace').decode()
+
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 2:
+            syslog.syslog("mail from: {0} {1}".format(f, moredata))
         self.fp = io.BytesIO()
         self.mailfrom = f
         t = parse_addr(f)
@@ -126,24 +139,38 @@ class dkimMilter(Milter.Base):
     @Milter.noreply
     def header(self, name, val):
         lname = name.lower()
+        if self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4:
+            if lname == 'content-transfer-encoding':
+                syslog.syslog("content-transfer-encodeing: {0}".format(val))
+            if lname == 'content-type':
+                syslog.syslog("content-type: {0}".format(val))
         if lname == 'dkim-signature':
-            if (milterconfig.get('Syslog') and
+            if (self.conf.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 1):
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog("{0}: {1}".format(name, val))
             self.has_dkim += 1
         if lname == 'from':
-            fname, self.author = parseaddr(val)
+            fname, self.author = parseaddr(idna.alabel(val))
             try:
                 self.fdomain = self.author.split('@')[1].lower()
             except IndexError as er:
                 pass # self.author was not a proper email address
-            if (milterconfig.get('Syslog') and
+            # This keeps non-ascii characters out of the From domain
-                    milterconfig.get('debugLevel') >= 1):
+            try:
+                self.fdomain = str(codecs.encode(self.fdomain, 'ascii', 'replace'), 'ascii', 'ignore')
+            except TypeError:
+                self.fdomain = codecs.encode(self.fdomain, 'ascii', 'replace').decode('ascii','ignore')
+            if (self.conf.get('Syslog') and
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog("{0}: {1}".format(name, val))
         elif lname == 'authentication-results':
             self.arheaders.append(val)
         if self.fp:
             try:
+                if lname == 'from':
+                    # Non-ascii in email address localpart is legal, so this is a special case
+                    self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'UTF-8', 'replace')))
+                else:
                     self.fp.write(b"%s: %s\n" % (codecs.encode(name, 'ascii'), codecs.encode(val, 'ascii')))
             except:
                 # Don't choke on header fields with non-ascii garbage in them.
@@ -175,42 +202,132 @@ class dkimMilter(Milter.Base):
                       .parse_value(FWS.sub('', val)))
                 if ar.authserv_id == self.AuthservID:
                     self.chgheader('authentication-results', i, '')
-                    if (milterconfig.get('Syslog') and
+                    if (self.conf.get('Syslog') and
-                            milterconfig.get('debugLevel') >= 1):
+                            self.conf.get('debugLevel') >= 1):
                         syslog.syslog('REMOVE: {0}'.format(val))
             except:
                 # Don't error out on unparseable AR header fiels
                 pass
         # Check and/or sign DKIM
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 4):
+            syslog.syslog('self.conf: {0}'.format(self.conf))
         self.fp.seek(0)
         txt = self.fp.read()
-        if milterconfig.get('Domain'):
+        self.get_identities_sign()
-            domain = milterconfig.get('Domain')
+        if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3):
-        else:
+            syslog.syslog('self.domain: {0}, self.fdomain: {1}, self.iequals: {2}'.format(self.domain, self.fdomain, self.iequals))
-            domain = ''
+        if ((self.fdomain in self.domain) and not self.conf.get('Mode') == 'v'
-        if milterconfig.get('SubDomains'):
-            self.fdomain = _get_parent_domain(self.fdomain, domain)
-        if ((self.fdomain in domain) and not milterconfig.get('Mode') == 'v'
                 and not self.external_connection):
+            if (self.conf.get('Syslog') and self.conf.get('debugLevel') >= 3):
+                syslog.syslog("Signing DKIM")
             self.sign_dkim(txt)
         if ((self.has_dkim) and (not self.internal_connection) and
-            (milterconfig.get('Mode') == 'v' or
+            (self.conf.get('Mode') == 'v' or
-             milterconfig.get('Mode') == 'sv')):
+             self.conf.get('Mode') == 'sv')):
             self.check_dkim(txt)
         if self.arresults:
             h = authres.AuthenticationResultsHeader(authserv_id=
                                                     self.AuthservID,
                                                     results=self.arresults)
             h = fold(codecs.encode(str(h), 'ascii'))
-            if (milterconfig.get('Syslog') and
+            if (self.conf.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 2):
+                    self.conf.get('debugLevel') >= 2):
                 syslog.syslog(codecs.decode(h, 'ascii'))
             name, val = codecs.decode(h, 'ascii').split(': ', 1)
             self.addheader(name, val, 0)
         return Milter.CONTINUE
 
+    # get parent domain to be signed for if fdomain is a subdomain
+    def get_parent_domain(self, fdomain, domains):
+        for domain in domains:
+            rhs = '.'+domain
+            # compare right hand side of fdomain against .domain
+            if fdomain[-len(rhs):] == rhs:
+                # return parent domain on match
+                syslog.syslog('domain: {0}'.format(domain))
+                return domain
+        # or return the fdomain itself
+        return fdomain
+
+    def get_identities_sign(self):
+        """Determine d= and i= identiies for signature"""
+        self.domain = []
+        self.iequals = None
+        try:
+            self.privkeyRSA = self.conf.get('privateRSA')
+        except:
+            self.privkeyRSA = ''
+        try:
+            self.privkeyEd25519 = self.conf.get('privateEd25519')
+        except:
+            self.privkeyEd25519 = ''
+        try:
+            self.selectorRSA = self.conf.get('Selector')
+        except:
+            self.selectorRSA = ''
+        try:
+            self.selectorEd25519 = self.conf.get('SelectorEd25519')
+        except:
+            self.selectorEd25519 = ''
+        if not self.domain and self.conf.get('Domain'):
+            self.domain = self.conf.get('Domain')
+        if self.conf.get('SubDomains'):
+            self.fdomain = self.get_parent_domain(self.fdomain, self.domain)
+        if self.conf.get('SigningTable'):
+            match = False
+            for dictkey, dictvalues in self.conf.get('SigningTable').items():
+                if dictkey == '%':
+                    self.domain.append(self.fdomain)
+                    match = True
+                elif len(dictkey.split('*')) == 1:
+                    if dictkey == self.author:
+                        self.domain.append(self.fdomain)
+                        match = True
+                else:
+                    if len(dictkey.split('*')) == 2:
+                        if dictkey.split('*')[1] == self.author[-len(dictkey.split('*')[1]):]:
+                            self.domain.append(self.fdomain)
+                            match = True
+                    self.domain.append(self.fdomain)
+                try:
+                    if len(dictvalues) == 2 and match:
+                        if dictvalues[0] =='%':
+                            self.iequals = codecs.encode('@' + self.fdomain)
+                        elif dictvalues[0][1:] == self.fdomain or self.get_parent_domain(dictvalues[0][1:], self.domain) == self.fdomain:
+                            self.iequals = codecs.encode(dictvalues[0])
+                except IndexError:
+                    pass
+                if match:
+                    #TODO add KeyTable stuffs here.
+                    keytablekey = dictvalues[-1] # Last value in the SigningTable row.
+                    if self.conf.get('privateRSATable'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateRSATable')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.domain.append(self.fdomain)
+                            self.selectorRSA = keytabledata[1]
+                            self.privkeyRSA = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    if self.conf.get('privateEd25519Table'):
+                        # Table data is a list of [ signing domain, selector, key ]
+                        keytabledata = self.conf.get('privateEd25519Table')[keytablekey]
+                        try:
+                            self.fdomain = keytabledata[0]
+                            self.domain.append(self.fdomain)
+                            self.selectorEd25519 = keytabledata[1]
+                            self.privkeyEd25519 = keytabledata[2]
+                        except:
+                            if (self.conf.get('Syslog')):
+                                    syslog.syslog('Error: Invalid KeyTable data {0}'.format(keytabledata))
+                    if (self.fdomain == '%'):
+                        self.fdomain = self.author.split('@')[1].lower()
+                    break
+
     def sign_dkim(self, txt):
-        canon = codecs.encode(milterconfig.get('Canonicalization'), 'ascii')
+        canon = codecs.encode(self.conf.get('Canonicalization'), 'ascii')
         canonicalize = []
         if len(canon.split(b'/')) == 2:
             canonicalize.append(canon.split(b'/')[0])
@@ -218,47 +335,52 @@ class dkimMilter(Milter.Base):
         else:
             canonicalize.append(canon)
             canonicalize.append(canon)
-            if (milterconfig.get('Syslog') and
+            if (self.conf.get('Syslog') and
-                    milterconfig.get('debugLevel') >= 1):
+                    self.conf.get('debugLevel') >= 1):
                 syslog.syslog('canonicalize: {0}'.format(canonicalize))
+        sign_headers = self.conf.get('SignHeaders')
+        if not sign_headers:
+            # None or empty. DKIM explicitly tests for None.
+            sign_headers = None
         try:
-            if privateRSA:
+            if self.privkeyRSA:
                 d = dkim.DKIM(txt)
-                h = d.sign(codecs.encode(milterconfig.get('Selector'), 'ascii'), codecs.encode(self.fdomain, 'ascii'),
+                h = d.sign(codecs.encode(self.selectorRSA, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
-                           codecs.encode(privateRSA, 'ascii'),
+                           codecs.encode(self.privkeyRSA, 'ascii'),
-                           canonicalize=(canonicalize[0],
+                           canonicalize=(canonicalize[0], canonicalize[1]),
-                                         canonicalize[1]))
+                           identity=self.iequals, include_headers=sign_headers)
                 name, val = h.split(b': ', 1)
                 self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
-                if (milterconfig.get('Syslog') and
+                if (self.conf.get('Syslog') and
-                    (milterconfig.get('SyslogSuccess')
+                    (self.conf.get('SyslogSuccess')
-                     or milterconfig.get('debugLevel') >= 1)):
+                     or self.conf.get('debugLevel') >= 1)):
                     syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
                                                   d.signature_fields.get(b'a').decode(),
                                                   d.signature_fields.get(b's').decode(),
                                                   d.domain.decode().lower()))
-            if privateEd25519:
+            if self.privkeyEd25519:
                 d = dkim.DKIM(txt)
-                h = d.sign(codecs.encode(milterconfig.get('SelectorEd25519'), 'ascii'), codecs.encode(self.fdomain, 'ascii'),
+                h = d.sign(codecs.encode(self.selectorEd25519, 'ascii'), codecs.encode(self.fdomain, 'ascii'),
-                           privateEd25519, canonicalize=(canonicalize[0],
+                           self.privkeyEd25519,
-                                                         canonicalize[1]),
+                           canonicalize=(canonicalize[0], canonicalize[1]),
+                           identity=self.iequals, include_headers=sign_headers,
                            signature_algorithm=b'ed25519-sha256')
                 name, val = h.split(b': ', 1)
                 self.addheader(codecs.decode(name, 'ascii'), codecs.decode(val, 'ascii').strip().replace('\r\n', '\n'), 0)
-                if (milterconfig.get('Syslog') and
+                if (self.conf.get('Syslog') and
-                    (milterconfig.get('SyslogSuccess')
+                    (self.conf.get('SyslogSuccess')
-                     or milterconfig.get('debugLevel') >= 1)):
+                     or self.conf.get('debugLevel') >= 1)):
                     syslog.syslog('{0}: {1} DKIM signature added (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
                                                   d.signature_fields.get(b'a').decode(),
                                                   d.signature_fields.get(b's').decode(),
                                                   d.domain.decode().lower()))
         except dkim.DKIMException as x:
-            if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog('DKIM: {0}'.format(x))
         except Exception as x:
-            if milterconfig.get('Syslog'):
+            if self.conf.get('Syslog'):
                 syslog.syslog("sign_dkim: {0}".format(x))
             raise
 
@@ -266,12 +388,16 @@ class dkimMilter(Milter.Base):
         res = False
         self.header_a = None
         for y in range(self.has_dkim):  # Verify _ALL_ the signatures
-            d = dkim.DKIM(txt)
+            d = dkim.DKIM(txt, minkey=self.conf.get('MinimumKeyBits'), timeout=self.conf.get('DNSTimeout'))
             try:
-                dnsoverride = milterconfig.get('DNSOverride')
+                dnsoverride = self.conf.get('DNSOverride')
                 if isinstance(dnsoverride, str):
+                    timeout = 5
+                    domain = self.fdomain
+                    def dnsfunc(domain, timeout=timeout, dnsoverride=dnsoverride):
+                        return dnsoverride
                     syslog.syslog("DNSOverride: {0}".format(dnsoverride))
-                    res = d.verify(idx=y, dnsfunc=lambda _x: dnsoverride)
+                    res = d.verify(idx=y, dnsfunc=dnsfunc)
                 else:
                     res = d.verify(idx=y)
                 algo = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
@@ -287,11 +413,11 @@ class dkimMilter(Milter.Base):
                                          .format(d.keysize, algo))
             except dkim.DKIMException as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
                     syslog.syslog('DKIM: {0}'.format(x))
             except Exception as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
                     syslog.syslog("check_dkim: Internal program fault while verifying: {0}".format(x))
             try:
                 # i= is optional and dkimpy is fine if it's not provided
@@ -303,15 +429,15 @@ class dkimMilter(Milter.Base):
                 self.header_a = codecs.decode(d.signature_fields.get(b'a'), 'ascii')
             except Exception as x:
                 self.dkim_comment = str(x)
-                if milterconfig.get('Syslog'):
+                if self.conf.get('Syslog'):
-                    syslog.syslog("check_dkim: Internal proram fuault extracting header a or d: {0}".format(x))
+                    syslog.syslog("check_dkim: Internal program fault extracting header a or d: {0}".format(x))
                 self.header_d = None
             if not self.header_a:
                 self.header_a = 'rsa-sha256'
             if res:
-                if (milterconfig.get('Syslog') and
+                if (self.conf.get('Syslog') and
-                        (milterconfig.get('SyslogSuccess') or
+                        (self.conf.get('SyslogSuccess') or
-                         milterconfig.get('debugLevel') >= 1)):
+                         self.conf.get('debugLevel') >= 1)):
                     syslog.syslog('{0}: {1} DKIM signature verified (s={2} '
                                   'd={3})'.format(self.getsymval('i'),
                                                   d.signature_fields.get(b'a').decode(),
@@ -319,15 +445,16 @@ class dkimMilter(Milter.Base):
                                                   d.domain.decode().lower()))
                 self.dkim_domain = d.domain.lower()
             else:
-                if milterconfig.get('DiagnosticDirectory'):
+                if self.conf.get('DiagnosticDirectory'):
+                    tempfile.tempdir = self.conf.get('DiagnosticDirectory')
                     fd, fname = tempfile.mkstemp(".dkim")
                     with os.fdopen(fd, "w+b") as fp:
                         fp.write(txt)
-                    if milterconfig.get('Syslog'):
+                    if self.conf.get('Syslog'):
                         syslog.syslog('DKIM: Fail (saved as {0})'
                                       .format(fname))
                 else:
-                    if milterconfig.get('Syslog'):
+                    if self.conf.get('Syslog'):
                         if d.domain:
                             syslog.syslog('DKIM: Fail ({0})'
                                           .format(d.domain.lower()))
@@ -350,41 +477,28 @@ class dkimMilter(Milter.Base):
             self.header_a = None
         return
 
-# get parent domain to be signed for if fdomain is a subdomain
-def _get_parent_domain(fdomain, domains):
-    for domain in domains:
-        rhs = '.'+domain
-        # compare right hand side of fdomain against .domain
-        if fdomain[-len(rhs):] == rhs:
-            # return parent domain on match
-            return domain
-    # or return the fdomain itself
-    return fdomain
-
 def main():
     # Ugh, but there's no easy way around this.
     global milterconfig
-    global privateRSA
-    global privateEd25519
-    privateRSA = False
-    privateEd25519 = False
     configFile = '/usr/local/etc/dkimpy-milter.conf'
     if len(sys.argv) > 1:
-        if sys.argv[1] in ('-?', '--help', '-h'):
+        if (sys.argv[1] in ('-?', '--help', '-h')) or len(sys.argv) == 3 or \
-            print('usage: dkimpy-milter [<configfilename>]')
+               (len(sys.argv) == 4 and sys.argv[2] != '-P'):
+            print('usage: dkimpy-milter [<configfilename> [-P <pidfile>]]')
             sys.exit(1)
         configFile = sys.argv[1]
     milterconfig = config._processConfigFile(filename=configFile)
+    if len(sys.argv) == 4:
+        if sys.argv[2] == '-P':
+            # Command line PID file argument overrides config file
+            milterconfig['PidFile'] = sys.argv[3]
     if milterconfig.get('Syslog'):
         facility = eval("syslog.LOG_{0}"
                         .format(milterconfig.get('SyslogFacility').upper()))
         syslog.openlog(os.path.basename(sys.argv[0]), syslog.LOG_PID, facility)
         setExceptHook()
     pid = write_pid(milterconfig)
-    if milterconfig.get('KeyFile'):
+    milterconfig = get_keys(milterconfig)
-        privateRSA = read_keyfile(milterconfig, 'RSA')
-    if milterconfig.get('KeyFileEd25519'):
-        privateEd25519 = read_keyfile(milterconfig, 'Ed25519')
     Milter.factory = dkimMilter
     Milter.set_flags(Milter.CHGHDRS + Milter.ADDHDRS)
     miltername = 'dkimpy-filter'
@@ -400,11 +514,11 @@ def main():
                     socketname = 'fd:3'
         if socketname is None:
             socketname = 'local:/var/run/dkimpy-milter/dkimpy-milter.sock'
-    drop_privileges(milterconfig)
     sys.stdout.flush()
     if milterconfig.get('Syslog'):
         syslog.syslog('dkimpy-milter starting:{0} user:{1}'
                       .format(pid, milterconfig.get('UserID')))
+    drop_privileges(milterconfig)
     Milter.runmilter(miltername, socketname, 240)
 
 if __name__ == "__main__":

dkimpy_milter/config.py

@@ -39,6 +39,7 @@ defaultConfigData = {
     'SyslogFacility': 'mail',
     'UMask': 0o07,
     'Mode': 'sv',
+    'MinimumKeyBits': 1024,
     'Socket': None,
     'PidFile': None,
     'UserID': 'dkimpy-milter',
@@ -49,7 +50,9 @@ defaultConfigData = {
     'MacroList': '',
     'MacroListVerify': '',
     'DNSOverride': None,
+    'DNSTimeout': 5,
     'SubDomains': False,
+    'SigningTable': None,
     'debugLevel': 0  # Undocumented config item for developer use
     }
 
@@ -85,17 +88,24 @@ class HostsDataset(object):
             if self.item[0] == '!':
                 self.item = item[1:]
                 self.negative = True
+            try:
                 try:
                     self.item = ipaddress.ip_address(str(self.item, "utf-8"))
+                except TypeError:
+                    self.item = ipaddress.ip_address(self.item)
                 if isinstance(self.item, ipaddress.IPv4Address):
                     self.isipv4 = True
                 elif isinstance(self.item, ipaddress.IPv6Address):
                     self.isipv6 = True
             except ValueError as e:
+                try:
                     try:
                         self.item = ipaddress.ip_network(str
                                                          (self.item, "utf-8"),
                                                          strict=False)
+                    except TypeError:
+                        self.item = ipaddress.ip_network(self.item,
+                                                         strict=False)
                     if isinstance(self.item, ipaddress.IPv4Network):
                         self.isipv4cidr = True
                     elif isinstance(self.item, ipaddress.IPv6Network):
@@ -111,7 +121,10 @@ class HostsDataset(object):
 
     def match(self, connectip):
         '''Check if the connect IP is part of the dataset'''
+        try:
             source = ipaddress.ip_address(str(connectip, "utf-8"))
+        except TypeError:
+            source = ipaddress.ip_address(connectip)
         for item in self.dataset:
             if item.isdomain or item.ishostname:
                 result = self.matchname(source)   # Match host/domains first
@@ -156,18 +169,24 @@ class HostsDataset(object):
         '''Get validated PTR name of IP address'''
         results = []
         s = Session()
-        ptrnames = s.dns(source.reverse_pointer, 'PTR')
+        ptrnames = s.dns(source.reverse_pointer, 'PTR', timeout=self.conf.get('DNSTimeout'))
         for name in ptrnames:
             if isinstance(source, ipaddress.IPv4Address):
                 ips = s.dns(name, 'A')
                 for ip in ips:
+                    try:
                         ip = ipaddress.IPv4Address(str(ip, 'UTF-8'))
+                    except TypeError:
+                        ip = ipaddress.IPv4Address(ip)
                     if ip == source:
                         results.append(name)
             if isinstance(source, ipaddress.IPv6Address):
                 ips = s.dns(name, 'AAAA')
                 for ip in ips:
+                    try:
                         ip = ipaddress.IPv6Address(str(ip, 'UTF-8'))
+                    except TypeError:
+                        ip = ipaddress.IPv6Address(ip)
                 if ip == source:
                     results.append(name)
         return results
@@ -260,10 +279,10 @@ def _make_authserv_id(as_id):
         as_id = socket.gethostname()
     return as_id
 
-
 def _dataset_to_list(dataset):
     """Convert a dataset (as defined in dkimpymilter.8) and return a python
-       list of values."""
+       list of values.  For multiline datasets like KeyTable and SigningTable a
+       key : values dictionary is returned"""
     if not isinstance(dataset, str):
         # If it was a csl with more than one value, it's already a list, we
         # only need to remove the name from the first value.
@@ -273,28 +292,45 @@ def _dataset_to_list(dataset):
             dataset[dataset.index(item)] = item.strip().strip(',')
         return dataset
     elif isinstance(dataset, str):
-        if dataset[0] == '/' or dataset[:5] == 'file:':
+        if dataset[0] == '/' or dataset[:5] == 'file:' or dataset[:7] == 'refile:':
-            # This is a flat file dataset
+            # This is a flat file dataset, which are key value:value stores
             ds = []
-            if dataset[0] == '/':
+            dsd = {}
+            if dataset[0] == '/' or dataset[:2] == './' or dataset[:3] == '../':
                 dsname = dataset
-            if dataset[:5] == 'file:':
+            elif dataset[:5] == 'file:':
                 dsname = dataset[5:]
+            elif dataset[:7] == 'refile:':
+                dsname = dataset[7:]
             dsf = open(dsname, 'r')
             for line in dsf.readlines():
                 if line[0] != '#':
+                    if len(line.split()) == 1:
                         if len(line.split(':')) == 1:
                             ds.append(line.strip())
                         else:
                             for element in line.split(':'):
                                 ds.append(element.strip().strip(':'))
+                    elif len(line.split()) == 2: # key value:value:value
+                        key, values = line.split()
+                        values = values.split(':')
+                        dsd.update({key:values})
             dsf.close()
+            if ds:
                 return ds
+            elif dsd:
+                return dsd
         # If it's a str and csl, it has one value and we return a list
         if dataset[:4] == 'csl:':
-            return [dataset[4:].strip().strip(',')]
+            datalist = dataset[4:].split(',')
+            for item in datalist:
+                datalist[datalist.index(item)] = item.strip().strip(',')
+            return datalist
         else:
-            return [dataset.strip().strip(',')]
+            datalist = dataset.split(',')
+            for item in datalist:
+                datalist[datalist.index(item)] = item.strip().strip(',')
+            return datalist
         if dataset[-3:] == '.db' or dataset[:3] == 'db:':
             #  This is a Sleepycat (Oracle) DB  dataset, which we dont support
             raise dkim.ParameterError('Unsupported dataset db datase: {0}'
@@ -324,15 +360,19 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         'SyslogSuccess': 'bool',
         'UMask': 'int',
         'Mode': 'str',
+        'MinimumKeyBits': 'int',
         'Socket': 'str',
         'PidFile': 'str',
         'UserID': 'str',
         'Domain': 'dataset',
         'SubDomains': 'bool',
         'KeyFile': 'str',
+        'KeyTable': 'dataset',
         'KeyFileEd25519': 'str',
+        'KeyTableEd25519': 'dataset',
         'Selector': 'str',
         'SelectorEd25519': 'str',
+        'SigningTable': 'dataset',
         'Canonicalization': 'str',
         'InternalHosts': 'dataset',
         'IntHosts': 'bool',
@@ -340,7 +380,9 @@ def _readConfigFile(path, configData=None, configGlobal={}):
         'MacroList': 'dataset',
         'MacroListVerify': 'dataset',
         'DNSOverride': 'str',
-        'debugLevel': 'int'
+        'DNSTimeout': 'int',
+        'debugLevel': 'int',
+        'SignHeaders': 'dataset'
         }
 
     #  check to see if it's a file
@@ -405,6 +447,10 @@ def _readConfigFile(path, configData=None, configGlobal={}):
             else:
                 configData[name] = str(value)
         elif conversion == 'int':
+            if name == 'MinimumKeyBits':
+                if int(value) == 0:
+                    # Odd inheritence from OpenDKIM where value of 0 means use default.
+                    value = configData.get(name)
             configData[name] = int(value)
         elif conversion == 'dataset':
             configData[name] = _dataset_to_list(value)
@@ -415,8 +461,14 @@ def _readConfigFile(path, configData=None, configGlobal={}):
     fp.close()
     try:
         configData['AuthservID'] = _make_authserv_id(configData.get('AuthservID', 'HOSTNAME'))
+    except Exception as e:
+        syslog.syslog("Could not make AuthservID: {}".format(e))
+        pass
+
+    try:
         configData['IntHosts'] = HostsDataset(configData['InternalHosts'])
-    except:
+    except Exception as e:
+        syslog.syslog("Could not make HostDataset from InternalHosts: {}".format(e))
         pass
 
     return(configData)

dkimpy_milter/dnsplug.py

@@ -88,7 +88,7 @@ class Session(object):
         result = self.dns(cname, qtype, cnames=cnames)
     return result
 
-def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
+def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=5):
     try:
 	# FIXME: To be thread safe, we create a fresh DnsRequest with
 	# each call.  It would be more efficient to reuse
@@ -114,11 +114,11 @@ def DNSLookup_pydns(name, qtype, tcpfallback=True, timeout=30):
     except IOError as x:
         raise DNS.DNSError('DNS: ' + str(x))
 
-def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=30):
+def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=5):
   retVal = []
   try:
     # FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
-    answers = dns.resolver.query(name, qtype)
+    answers = dns.resolver.query(name, qtype, raise_on_no_answer=False, lifetime=timeout)
     for rdata in answers:
       if qtype == 'A' or qtype == 'AAAA':
         retVal.append(((name, qtype), rdata.address))
@@ -127,7 +127,7 @@ def DNSLookup_dnspython(name,qtype,tcpfallback=True,timeout=30):
       elif qtype == 'PTR':
         retVal.append(((name, qtype), rdata.target.to_text(True)))
       elif qtype == 'TXT' or qtype == 'SPF':
-        retVal.append(((name, qtype), rdata.strings))
+        retVal.append(((name, qtype), list(rdata.strings)))
   except dns.resolver.NoAnswer:
     pass
   except dns.resolver.NXDOMAIN:

dkimpy_milter/util.py

@@ -149,13 +149,9 @@ def write_pid(milterconfig):
     return pid
 
 
-def read_keyfile(milterconfig, keytype):
+def read_keyfile(keyfile, milterconfig):
     """Read private key from file."""
     import syslog
-    if keytype == "RSA":
-        keyfile = milterconfig.get('KeyFile')
-    if keytype == "Ed25519":
-        keyfile = milterconfig.get('KeyFileEd25519')
     try:
         f = open(keyfile, 'r')
         keylist = f.readlines()
@@ -170,25 +166,36 @@ def read_keyfile(milterconfig, keytype):
         key += line
     return key
 
-def read_keytable(milterconfig, tabletype):
+def read_keytable(tabledict, milterconfig):
     """Read keytables into in memory configuration data so all keys are read
-    before priviledges are dropped."""
+    before priviledges are dropped.
+    When loaded, tabeldict is a dict:
+        {searchkey: [donamin, selector, key]}
+    If key is a file (startswith('/'), then the key is returned in its place."""
+    import dkim
     import syslog
-    if tabletype == "RSA":
+    for dictkey, values in tabledict.items():
-        tablefile = milterconfig.get('KeyTable')
+        if values[-1][:1] == '/' or values[-1][:2] == './' or values[-1][:3] == '../':
-    if tabletype == "Ed25519":
+            key = read_keyfile(values[-1], milterconfig)
-        tablefile = milterconfig.get('KeyTableEd25519')
+        tabledict[dictkey] = [values[0], values[1], key]
-    if milterconfig.get(tablefile):
+    return tabledict
-        keytabledata = []
-        try:
-            f = open(milterconfig.get(tablefile))
-            for row in f:
-                keytabledata.append(row) 
-            f.close()
-        except IOError as e:
-            if milterconfig.get('Syslog'):
-                syslog.syslog('Unable to read keytable {0}.  IOError: {1}'
-                              .format(tablefile, e))
-            raise
 
-    return keytabledata
+def get_keys(milterconfig):
+    """Read keys (table or file) into memory before dropping priviledges"""
+    milterconfig['privateRSA'] = False
+    milterconfig['privateRSATable'] = False
+    milterconfig['privateEd25519'] = False
+    milterconfig['privateEd25519Table'] = False
+    if milterconfig.get('KeyTable'):
+        milterconfig['privateRSATable'] = read_keytable(milterconfig.get('KeyTable'),
+                                                        milterconfig)
+    elif milterconfig.get('KeyFile'):
+        milterconfig['privateRSA'] = read_keyfile(milterconfig.get('KeyFile'),
+                                                  milterconfig)
+    if milterconfig.get('KeyTableEd25519'):
+        milterconfig['privateEd25519Table'] = read_keytable(milterconfig.get('KeyTableEd25519'),
+                                                            milterconfig)
+    elif milterconfig.get('KeyFileEd25519'):
+        milterconfig['privateEd25519'] = read_keyfile(milterconfig.get('KeyFileEd25519'),
+                                                      milterconfig)
+    return milterconfig

etc/dkimpy-milter.conf

@@ -1,6 +1,5 @@
 # This is a basic configuration that can easily be adapted to suit a standard
-# installation. For more advanced options, see dkimpy-milter.conf(5) and/or
+# installation. For more advanced options, see dkimpy-milter.conf(5).
-# /usr/share/doc/dkimpy-milter/examples/opendkim.conf.sample.
 
 # Log to syslog
 Syslog			yes
@@ -9,18 +8,16 @@ Syslog			yes
 # privileged user (e.g. Postfix)
 UMask			007
 
-# Sign for example.com with key in /etc/dkimkeys/dkim.key using
+# Sign for example.com with key in /usr/local/etc/dkimkeys/dkim.key using
 # selector '2007' (e.g. 2007._domainkey.example.com)
 #Domain			example.com
-#KeyFile			/etc/mail/dkim.key
+#KeyFile		/usr/local/etc/mail/dkim.key
 #Selector		default
 
 # Commonly-used options; the commented-out versions show the defaults.
 #Canonicalization	relaxed/simple
 #Mode			sv
 
-# Socket local:/var/run/dkimpy-milter/dkimpy-milter.sock
-#
 # ##  Socket socketspec
 # ##
 # ##  Names the socket where this filter should listen for milter connections
@@ -30,10 +27,12 @@ UMask			007
 # ##  inet:port                   to listen on all interfaces
 # ##  local:/path/to/socket       to listen on a UNIX domain socket
 #
-Socket			inet:8892@localhost
+#Socket local:/run/dkimpy-milter/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
 
 ##  PidFile filename
-###      default /var/run/dkimpy-milter/dkimpy-milter.pid
+###      default /run/dkimpy-milter/dkimpy-milter.pid
 ###
 ###  Name of the file where the filter should write its pid before beginning
 ###  normal operations.

etc/dkimpy-milter.conf.in

@@ -0,0 +1,48 @@
+# This is a basic configuration that can easily be adapted to suit a standard
+# installation. For more advanced options, see dkimpy-milter.conf(5).
+
+# Log to syslog
+Syslog			yes
+
+# Required to use local socket with MTAs that access the socket as a non-
+# privileged user (e.g. Postfix)
+UMask			007
+
+# Sign for example.com with key in @SYSCONFDIR@/dkimkeys/dkim.key using
+# selector '2007' (e.g. 2007._domainkey.example.com)
+#Domain			example.com
+#KeyFile		@SYSCONFDIR@/mail/dkim.key
+#Selector		default
+
+# Commonly-used options; the commented-out versions show the defaults.
+#Canonicalization	relaxed/simple
+#Mode			sv
+
+# ##  Socket socketspec
+# ##
+# ##  Names the socket where this filter should listen for milter connections
+# ##  from the MTA.  Required.  Should be in one of these forms:
+# ##
+# ##  inet:port@address           to listen on a specific interface
+# ##  inet:port                   to listen on all interfaces
+# ##  local:/path/to/socket       to listen on a UNIX domain socket
+#
+#Socket local:@RUNSTATEDIR@/dkimpy-milter.sock
+#
+#Socket			inet:8892@localhost
+
+##  PidFile filename
+###      default /run/dkimpy-milter/dkimpy-milter.pid
+###
+###  Name of the file where the filter should write its pid before beginning
+###  normal operations.
+#
+PidFile			@RUNSTATEDIR@/dkimpy-milter.pid
+
+##  Userid userid
+###      default dkimpy-milter
+###
+###  Change to user "userid" before starting normal operation?  May include
+###  a group ID as well, separated from the userid by a colon.
+#
+UserID			dkimpy-milter

man/dkimpy-milter.8

@@ -170,22 +170,22 @@ are ignored, and the hash ("#") character denotes the start of a comment.
 If a value contains multiple entries, the entries should be separated by
 colons.
 .TP
-.I c)
+.I b)
-If the string begins with "db:" and the program was compiled with
+If the string begins with "refile:", then the remainder of the string is
-Sleepycat DB support, then the remainder of the string is presumed to
+presumed to specify a file that contains a set of patterns, one per line,
-identify a Sleepycat database containing keys and corresponding values.
+and their associated values.  The pattern is taken as the start of the line
-These may be used only to test for membership in the data set, or for
+to the first whitespace, and the portion after that whitespace is taken as
-storing keys and corresponding values.  If a value contains multiple entries,
+the value to be used when that pattern is matched.  Patterns are simple
-the entries should be separated by colons. [Not implemented yet]
+wildcard patterns, matching all text except that the asterisk ("*") character
-.TP
+is considered a wildcard.  If a value contains multiple entries, the entries
-.I h)
+should be separated by colons.
-If the string contains none of these prefixes but ends with ".db", it
-is presumed to be a Sleepycat DB as described above (if support for same
-is compiled in).  [Not implemented yet]
 .TP
 .I i)
 If the string contains none of these prefixes but starts with a slash ("/")
-character, it is presumed to be a flat file as described above.
+character, or "./" or "../", it is presumed to be a flat file as described
+above. Note: In OpenDKIM "./" and "../" only apply to KeyTable, but for
+dkimpy-milter it is generally applicable and KeyTable specification is not
+a special case.
 .TP
 .I j)
 If the string begins with "csl:", the string is treated as a comma-separated
@@ -204,8 +204,9 @@ pairs as described above.
 .TP
 See
 .I dkimpy-milter.conf (5)
-information about available options.  Unlike OpenDKIM, dkimpy-milter does not
+for information about available options.  Unlike OpenDKIM, with the exception of
-support command line option switches.
+\-P for the pidfile and specifying the configuration file to use,
+dkimpy-milter does not support command line option switches.
 
 When signing a message, a
 .I DKIM-Signature:

man/dkimpy-milter.conf.5

@@ -127,11 +127,11 @@
 .rm #[ #] #H #V #F C
 .\" ========================================================================
 .\"
-.TH dkimpy-milter.conf 5 "2018-02-12"
+.TH dkimpy-milter.conf 5 "2019-04-25"
 .SH "NAME"
 dkimpy-milter \- Python milter for DKIM signing and validation
 .SH "VERSION"
-1\.1\.0
+1\.2\.0
 
 .SH "DESCRIPTION"
 .I dkimpy-milter(8)
@@ -152,13 +152,14 @@ the value is processed.  For positive values, the following are accepted:
 "F", "f", "N", "n", "0".
 
 The provided setup.py installs this configuration file in /etc or
-/usr/local/etc.
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
 
 Command line invocation of parameters as is done by OpenDKIM is not supported.
 
 .SH "USAGE"
 Usage:
-  dkimpy-milter [/etc/dkimpy-milter.conf]
+  dkimpy-milter [/usr/local/etc/dkimpy-milter/dkimpy-milter.conf]
 
 .SH "OTHER DOCUMENTATION"
 This documentation assumes you have read Postfix's README_FILES/MILTER_README
@@ -210,9 +211,11 @@ header and the second to the body.
 .TP
 .I DiagnosticDirectory (string)
 Directory into which to write diagnostic reports when message verification
-fails.  If not set (the default), these files are not generated.  [Unlike
+fails.  If not set (the default), these files are not generated.  The
-OpenDKIM, this applies to all messages, not just  on messages bearing a "z=" tag
+directory must exist, dkimpy-milter will not create it and an error will be
-because dkimpy does not yet support "z=".]
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
 
 .TP
 .I Domain (dataset)
@@ -221,8 +224,6 @@ domains will be verified rather than being signed.
 
 This parameter is not required if a
 .I SigningTable
-or
-.I SigningTableEd25519
 is in use; in that case, the list of signed domains is implied by the
 lines in that file.
 
@@ -243,6 +244,10 @@ Naturally, providing a value here overrides the default, so if mail from
 127.0.0.1 should be signed, the list provided here should include that
 address explicitly. [PeerList NOT IMPLEMENTED]
 
+Mail sent via connections from InternalHosts will not have any existing DKIM
+signatures verified.  This is not overridden by MacroList or Mode.  If the
+Mode is 'v', then no actions will be performed.
+
 .TP
 .I KeyFile (string)
 Gives the location of a PEM-formatted private key to be used for RSA signing
@@ -260,13 +265,15 @@ is defined.
 
 .TP
 .I KeyTable (dataset)
-Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: direct specification of keys in the table as is done by OpenDKIM is not supported.
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
 
 .TP
 .I KeyTableEd25519 (dataset)
-Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: direct specification of keys in the table as is done by OpenDKIM is not support
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
-ed.
 
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
 
 .TP
 .I MacroList (dataset)
@@ -295,6 +302,10 @@ at the time the filter receives a connection from the MTA and its availability
 depends upon the version of milter used to compile the filter and the version
 of the MTA making the connection.
 
+Mail sent via connections where macros that are in MacroList are provided
+will not have any existing DKIM signatures verified.  If the Mode is 'v', then
+no actions will be performed.
+
 .TP
 .I MacroListVerify (dataset)
 Defines a set of MTA-provided
@@ -305,6 +316,10 @@ Entries in this data set follow the same form as those of the
 .I MacroList
 option above.  [this option is not inhereted from OpenDKIM]  
 
+Mail sent via connections where macros that are in MacroListVerify are
+provided will be not DKIM signed.  If the Mode is 's', then no actions will
+be performed.
+
 .TP
 .I Mode (string)
 Selects operating modes.  The string is a concatenation of characters that
@@ -324,6 +339,35 @@ be set:
 (a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
 (b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
 
+The action to sign or verify is also affected by the InternalHosts, MacroList,
+and MacroListVerify options.  Those options may preclude signing or
+verification in some cases, but will not enable signing or verifying if not
+allowed by Mode.
+
+.TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
 .TP 
 .I DNSOverride (string)
 Provide a text string that a verifying milter should use instead of
@@ -331,6 +375,12 @@ consulting the DNS on each message.  This is useful primarily for
 testing purposes in environments where it is awkward to modify the
 system DNS resolution.  It should not be used in production.
 
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
+
 .TP
 .I PeerList (dataset)
 Identifies a set of "peers" that identifies clients whose connections
@@ -382,31 +432,31 @@ This parameter is ignored if a
 is defined.
 
 .TP
-.I SigningTable (dataset)
+.I SignHeaders (dataset)
-
+Specifies the set of header fields that should be included when generating
-Defines a table used to select one or more signatures to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+signatures.  If the list omits any header field that is mandated by the DKIM
-
+specification, those fields are implicitly added.  By default, those fields
-If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
-
+will be signed by the filter.  See the
-If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field. Entries are checked in the order in which they appear in the file. ["refile support not implemented"].
+.I OmitHeaders
-
+configuration option for more information about the format and interpretation
-For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+of this field.
-
-In any case, only the first match is applied.
 
 .TP
-.I SigningTableEd25519 (dataset)
+.I SigningTable (dataset)
 
-Defines a table used to select one or more signatures to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+Defines a table used to select one or more signing identities to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
 
 If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
 
-If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field. Entries are checked in the order in which they appear in the file. ["refile support not implemented"].
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
 
 For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
 
 In any case, only the first match is applied.
 
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
 .TP
 .I Socket (string)
 Specifies the socket that should be established by the filter to receive
@@ -484,11 +534,32 @@ unless an alternate
 .I group
 is specified.
 
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
+Default location of this file.
+
 .SH "AUTHORS"
 \ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
-It is based on dkimpy-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
 Copyright (c) 2013-2015 Stuart D. Gathman
-Copyright (c) 2018 Scott Kitterman <scott@kitterman.com>.  
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
 .PP
 This man-page was created by Scott Kitterman <scott@kitterman.com>.
 
@@ -502,4 +573,4 @@ See LICENSE.
 
 Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
 of the package.
-Copyright (c) 2018, Scott Kitterman <scott@kitterman.com>
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

man/dkimpy-milter.conf.5.in

@@ -0,0 +1,576 @@
+\"
+.\" Standard preamble:
+.\" ========================================================================
+.de Sh \" Subsection heading
+.br
+.if t .Sp
+.ne 5
+.PP
+\fB\\$1\fR
+.PP
+..
+.de Sp \" Vertical space (when we can't use .PP)
+.if t .sp .5v
+.if n .sp
+..
+.de Vb \" Begin verbatim text
+.ft CW
+.nf
+.ne \\$1
+..
+.de Ve \" End verbatim text
+.ft R
+.fi
+..
+.\" Set up some character translations and predefined strings.  \*(-- will
+.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
+.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
+.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
+.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
+.\" nothing in troff, for use with C<>.
+.tr \(*W-
+.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
+.ie n \{\
+.    ds -- \(*W-
+.    ds PI pi
+.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
+.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
+.    ds L" ""
+.    ds R" ""
+.    ds C` ""
+.    ds C' ""
+'br\}
+.el\{\
+.    ds -- \|\(em\|
+.    ds PI \(*p
+.    ds L" ``
+.    ds R" ''
+'br\}
+.\"
+.\" If the F register is turned on, we'll generate index entries on stderr for
+.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index
+.\" entries marked with X<> in POD.  Of course, you'll have to process the
+.\" output yourself in some meaningful fashion.
+.if \nF \{\
+.    de IX
+.    tm Index:\\$1\t\\n%\t"\\$2"
+..
+.    nr % 0
+.    rr F
+.\}
+.\"
+.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
+.\" way too many mistakes in technical documents.
+.hy 0
+.if n .na
+.\"
+.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
+.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
+.    \" fudge factors for nroff and troff
+.if n \{\
+.    ds #H 0
+.    ds #V .8m
+.    ds #F .3m
+.    ds #[ \f1
+.    ds #] \fP
+.\}
+.if t \{\
+.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
+.    ds #V .6m
+.    ds #F 0
+.    ds #[ \&
+.    ds #] \&
+.\}
+.    \" simple accents for nroff and troff
+.if n \{\
+.    ds ' \&
+.    ds ` \&
+.    ds ^ \&
+.    ds , \&
+.    ds ~ ~
+.    ds /
+.\}
+.if t \{\
+.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
+.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
+.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
+.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
+.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
+.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
+.\}
+.    \" troff and (daisy-wheel) nroff accents
+.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
+.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
+.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
+.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
+.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
+.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
+.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
+.ds ae a\h'-(\w'a'u*4/10)'e
+.ds Ae A\h'-(\w'A'u*4/10)'E
+.    \" corrections for vroff
+.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
+.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
+.    \" for low resolution devices (crt and lpr)
+.if \n(.H>23 .if \n(.V>19 \
+\{\
+.    ds : e
+.    ds 8 ss
+.    ds o a
+.    ds d- d\h'-1'\(ga
+.    ds D- D\h'-1'\(hy
+.    ds th \o'bp'
+.    ds Th \o'LP'
+.    ds ae ae
+.    ds Ae AE
+.\}
+.rm #[ #] #H #V #F C
+.\" ========================================================================
+.\"
+.TH dkimpy-milter.conf 5 "2019-04-25"
+.SH "NAME"
+dkimpy-milter \- Python milter for DKIM signing and validation
+.SH "VERSION"
+1\.2\.0
+
+.SH "DESCRIPTION"
+.I dkimpy-milter(8)
+implements the
+.B DKIM
+specification for signing and verifying e-mail messages on a per-domain
+basis.  This file is its configuration file.
+
+Blank lines are ignored.  Lines containing a hash ("#") character are
+truncated at the hash character to allow for comments in the file.
+
+Other content should be the name of a parameter, followed by white space,
+followed by the value of that parameter, each on a separate line.
+
+For parameters that are Boolean in nature, only the first byte of
+the value is processed.  For positive values, the following are accepted:
+"T", "t", "Y", "y", "1".  For negative values, the following are accepted:
+"F", "f", "N", "n", "0".
+
+The provided setup.py installs this configuration file in /etc or
+/usr/local/etc based on the value of expand sysconfigdir= used when the
+package was installed.
+
+Command line invocation of parameters as is done by OpenDKIM is not supported.
+
+.SH "USAGE"
+Usage:
+  dkimpy-milter [@CONFDIR@/dkimpy-milter.conf]
+
+.SH "OTHER DOCUMENTATION"
+This documentation assumes you have read Postfix's README_FILES/MILTER_README
+(or Sendmail equivalent) and are generally familiar with Domain Keys Identified
+Mail (DKIM).  See RFC 6376 for details.
+
+.SH "SYNOPSIS"
+
+dkimpy-milter operates with a default installed configuration file and 
+set of default configuration options that are used if the configuration file
+cannot be found.  These options can be changed by changing the installed 
+configuration files.  For users transitioning from OpenDKIM, OpenDKIM config
+files can be used directly.  Not all OpenDKIM options are supported.  If an
+unsupported option from OpenDKIM is specified, an error will be raised.
+
+.SH "DESCRIPTION"
+
+Configuration options are described here and in the configuration file 
+provided with the package.  The provided setup.py installs this configuration 
+file in /etc or /usr/local/etc.
+
+.SH "OPTIONS"
+
+.TP
+.I AuthservID (string)
+Sets the "authserv-id" to use when generating the Authentication-Results:
+header field after verifying a message.  The default is to use the name of
+the MTA processing the message.  If the string "HOSTNAME" is provided, the
+name of the host running the filter (as returned by the
+.I gethostname(3)
+function) will be used.
+
+.TP
+.I Canonicalization (string)
+Selects the canonicalization method(s) to be used when signing messages.
+When verifying, the message's DKIM-Signature: header field specifies
+the canonicalization method.  The recognized values are
+.I relaxed
+and
+.I simple
+as defined by the DKIM specification.  The default is
+.I relaxed
+/
+.I simple.
+The value may include two different canonicalizations separated by a
+slash ("/") character, in which case the first will be applied to the
+header and the second to the body.
+
+.TP
+.I DiagnosticDirectory (string)
+Directory into which to write diagnostic reports when message verification
+fails.  If not set (the default), these files are not generated.  The
+directory must exist, dkimpy-milter will not create it and an error will be
+raised if it does not.  [Unlike OpenDKIM, this applies to all messages, not
+just  on messages bearing a "z=" tag because dkimpy does not yet support
+"z=" processing.]
+
+.TP
+.I Domain (dataset)
+A set of domains whose mail should be signed by this filter.  Mail from other
+domains will be verified rather than being signed.
+
+This parameter is not required if a
+.I SigningTable
+is in use; in that case, the list of signed domains is implied by the
+lines in that file.
+
+This parameter is ignored if a
+.I KeyTable
+or
+.I KeyTableD25119
+is defined.
+
+.TP
+.I InternalHosts (dataset)
+Identifies a set internal hosts whose mail should be signed rather
+than verified.  Entries in this data set follow the same form as those of
+the
+.I PeerList
+option below.  If not specified, the default of "127.0.0.1" is applied.
+Naturally, providing a value here overrides the default, so if mail from
+127.0.0.1 should be signed, the list provided here should include that
+address explicitly. [PeerList NOT IMPLEMENTED]
+
+Mail sent via connections from InternalHosts will not have any existing DKIM
+signatures verified.  This is not overridden by MacroList or Mode.  If the
+Mode is 'v', then no actions will be performed.
+
+.TP
+.I KeyFile (string)
+Gives the location of a PEM-formatted private key to be used for RSA signing
+all messages.  Ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I KeyFileEd25519 (string)
+Gives the location of a Ed25519 private key to be used for Ed25519 signing
+all messages.  File is the Base64 encoded output of RFC 8032 Ed25519 private Key
+generation (as used in dkimpy).  Ignored if a 
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I KeyTable (dataset)
+Gives the location of a file mapping key names to RSA signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I KeyTableEd25519 (dataset)
+Gives the location of a file mapping key names to Ed25519 signing keys. If present, overrides any KeyFile setting in the configuration file. The data set named here maps each key name to three values: (a) the name of the domain to use in the signature’s "d=" value; (b) the name of the selector to use in the signature’s "s=" value; and (c) the  path to a file containing a private key. If the first value consists solely of a percent sign ("%") character, it will be replaced by the apparent domain of the sender when generating a signature. The third value must start with a slash ("/") character, or "./" or "../" to indicate it refers to a file from which the private key should be read.  The SigningTable (see below) is used to select records from this table to be used to add signatures based on the message sender.  NOTE: There is a limitation of the current implementation that a private key can't be directly included in the file if it starts with '/', './', or '../'.  If you have such a key, you may store it in a file and reference the file in the table.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I MacroList (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be a
+local user and therefore whether or not the message should be signed.  If
+a
+.I value
+is specified matching a macro name in the data set, the value of the macro
+must match a value specified (matching is case-sensitive), otherwise the
+macro must be defined but may contain any value.  The set is empty by
+default, meaning macros are not considered when making the sign-verify
+decision.  The general format of the value is
+.I value1[|value2[|...]];
+if one or more value is defined then the macro must be set to one of the
+listed values, otherwise the macro must be set but can contain any
+value.
+
+In order for the macro and its value to be available to the filter for
+checking, the MTA must send it during the protocol exchange.  This is either
+accomplished via manual configuration of the MTA to send the desired macros
+or, for MTA/filter combinations that support the feature, the filter can
+request those macros that are of interest.  The latter is a feature negotiated
+at the time the filter receives a connection from the MTA and its availability
+depends upon the version of milter used to compile the filter and the version
+of the MTA making the connection.
+
+Mail sent via connections where macros that are in MacroList are provided
+will not have any existing DKIM signatures verified.  If the Mode is 'v', then
+no actions will be performed.
+
+.TP
+.I MacroListVerify (dataset)
+Defines a set of MTA-provided
+.I macros
+that should be checked to see if the sender has been determined to be an
+external source and therefore whether or not the message should be signed.
+Entries in this data set follow the same form as those of the
+.I MacroList
+option above.  [this option is not inhereted from OpenDKIM]  
+
+Mail sent via connections where macros that are in MacroListVerify are
+provided will be not DKIM signed.  If the Mode is 's', then no actions will
+be performed.
+
+.TP
+.I Mode (string)
+Selects operating modes.  The string is a concatenation of characters that
+indicate which mode(s) of operation are desired.  Valid modes are
+.I s
+(signer) and
+.I v
+(verifier).  The default is
+.I sv
+except in test mode (see the
+.I opendkim(8)
+man page)
+in which case the default is
+.I v.
+When signing mode is enabled, one of the following combinations must also
+be set:
+(a) Domain, KeyFile, Selector, no KeyTable, no SigningTable;
+(b) KeyTable, SigningTable, no Domain, no KeyFile, no Selector;
+
+The action to sign or verify is also affected by the InternalHosts, MacroList,
+and MacroListVerify options.  Those options may preclude signing or
+verification in some cases, but will not enable signing or verifying if not
+allowed by Mode.
+
+.TP
+.I MinimumKeyBits (integer)
+Establishes a minimum key size for acceptable RSA signatures.  Signatures with
+smaller key sizes, even if they otherwise pass DKIM validation, will me marked
+as invalid.  The default is 1024, which accepts all signatures.  A value of
+0 causes the default to be used. Not Applicable to ed25519 signatures.
+
+.TP
+.I OmitHeaders (dataset)
+Specifies a set of header fields that should be omitted when generating
+signatures.  If an entry in the list names any header field that is mandated
+by the DKIM specification, the entry is ignored.  A set of header fields is
+listed in the DKIM specification (RFC6376, Section 5.4) as "SHOULD NOT" be
+signed; the default list for this parameter contains those fields
+(Return-Path, Received, Comments, Keywords, Bcc, Resent-Bcc and
+DKIM-Signature).  To omit no headers, simply use the string "." (or any
+string that will match no header field names).
+Specifying a list with this parameter replaces the default entirely, unless
+one entry is "*" in which case the list is interpreted as a delta to the
+default; for example, "*,+foobar" will use the entire default list plus
+the name "foobar", while "*,-Bcc" would use the entire default list except
+for the "Bcc" entry. [OmitHeaders NOT IMPLEMENTED - included for reference
+only]
+
+.TP 
+.I DNSOverride (string)
+Provide a text string that a verifying milter should use instead of
+consulting the DNS on each message.  This is useful primarily for
+testing purposes in environments where it is awkward to modify the
+system DNS resolution.  It should not be used in production.
+
+.TP
+.I DNSTimeout (integer)
+Sets the DNS timeout in seconds.  A value of 0 causes no wait (this is
+different than opendkim).  The default is 5.  See also the NOTES section
+below.
+
+.TP
+.I PeerList (dataset)
+Identifies a set of "peers" that identifies clients whose connections
+should be accepted without processing by this filter.  The set
+should contain on each line a hostname, domain name (e.g. ".example.com"),
+IP address, an IPv6 address (including an IPv4 mapped address), or a
+CIDR-style IP specification (e.g. "192.168.1.0/24").  An entry beginning
+with a bang ("!") character means "not", allowing exclusions of specific
+hosts that are otherwise members of larger sets.  Host and domain names are 
+matched first, then the IP or IPv6 address depending on the connection 
+type.  More precise entries are preferred over less precise ones, i.e. 
+"192.168.1.1" will match before "!192.168.1.0/24".  The text form of IPv6 
+addresses will be forced to lowercase when queried (RFC5952), so the contents
+of this data set should also use lowercase.  The IP address portion of an
+entry may optionally contain square brackets; both forms (with and without)
+will be checked. [PeerList NOT IMPLEMENTED - included for reference only]
+
+.TP
+.I PidFile (string)
+Specifies the path to a file that should be created at process start
+containing the process ID.  If not specified, no such file will be created.
+
+.TP
+.I Selector (string)
+Defines the name of the selector to be used when signing messages using RSA.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTable
+is defined.
+
+.TP
+.I SelectorEd25519 (string)
+Defines the name of the selector to be used when signing messages using Ed25519.
+See the
+.B DKIM
+specification for details.  Used only when signing with a single key;
+see the
+.I SigningTable
+parameter below for more information.
+
+This parameter is ignored if a
+.I KeyTableEd25519
+is defined.
+
+.TP
+.I SignHeaders (dataset)
+Specifies the set of header fields that should be included when generating
+signatures.  If the list omits any header field that is mandated by the DKIM
+specification, those fields are implicitly added.  By default, those fields
+listed in the DKIM specification as "SHOULD" be signed (RFC6376, Section 5.4)
+will be signed by the filter.  See the
+.I OmitHeaders
+configuration option for more information about the format and interpretation
+of this field.
+
+.TP
+.I SigningTable (dataset)
+
+Defines a table used to select a signing identity to apply to a message based on the address found in the From: header field. Keys in this table vary depending on the type of table used; values in this data set should include one field that contains a name found in the KeyTable (see above) that identifies which key should be used in generating the signature, and an optional second field naming the signer of the message that will be included in the "i=" tag in the generated signature. Note that the "i=" value will not be included in the signature if it conflicts with the signing domain (the "d=" value).
+
+If the first field contains only a "%" character, it will be replaced by the domain found in the From: header field. Similarly, within the optional second field, any "%" character will be replaced by the domain found in the From: header field.
+
+If this table specifies a regular expression file ("refile"), then the keys are wildcard patterns that are matched against the address found in the From: header field.  Entries are checked in the order in which they appear in the file.   Note: These are not true regular expressions.  The terminology is inherited from opendkim.  Only wildcards ("*") are supported.
+
+For all other database types, the full user@host is checked first, then simply host, then user@.domain (with all superdomains checked in sequence, so "foo.example.com" would first check "user@foo.example.com", then "user@.example.com", then "user@.com"), then .domain, then user@*, and finally *.
+
+In any case, only the first match is applied.
+
+See the COMPLEX SIGNING CONFIGURATIONS section of README.md for examples.
+
+.TP
+.I Socket (string)
+Specifies the socket that should be established by the filter to receive
+connections from
+.I postfix(1)
+in order to provide service.
+.I socketspec
+is in one of two forms:
+.I local:path,
+which creates a UNIX domain socket at the specified
+.I path,
+or
+.I inet:port[@host]
+or
+.I inet6:port[@host]
+which creates a TCP socket on the specified
+.I port
+and in the specified protocol family.  If the
+.I host
+is not given as either a hostname or an IP address, the socket will be
+listening on all interfaces.  A literal IP address must be enclosed in
+square brackets.  This option is mandatory in the configuration file.
+
+.TP
+.I SubDomains (Boolean)
+Sign subdomains of those listed by the
+.I Domain
+parameter as well as the actual domains.
+
+.TP
+.I Syslog (Boolean)
+Log via calls to
+.I syslog(3)
+any interesting activity.
+
+.TP
+.I SyslogFacility (string)
+Log via calls to
+.I syslog(3)
+using the named facility.  The facility names are the same as the ones
+allowed in
+.I syslog.conf(5).
+The default is "mail".
+
+.TP
+.I SyslogSuccess (Boolean)
+Log via calls to
+.I syslog(3)
+additional entries indicating successful signing or verification of
+messages.
+
+.TP
+.I UMask (integer)
+Requests a specific permissions mask to be used for file creation.
+This only really applies to creation of the socket when
+.I Socket
+specifies a UNIX domain socket, and to the
+.I PidFile
+(if any); temporary files are created by the
+.I mkstemp(3)
+function that enforces a specific file mode on creation regardless
+of the process umask.  See
+.I umask(2)
+for more information.
+
+.TP
+.I UserID (string)
+Attempts to become the specified userid before starting operations.
+The value is of the form
+.I userid[:group].
+The process will be assigned all of the groups and primary group ID of
+the named
+.I userid
+unless an alternate
+.I group
+is specified.
+
+.SH NOTES
+When using DNS timeouts (see the
+.I DNSTimeout
+option above), be sure not to use a timeout that is larger than the timeout
+being used for interaction between
+.I sendmail
+and the filter.  Otherwise, the MTA could abort a message while waiting for
+a reply from the filter, which in turn is still waiting for a DNS reply.  This
+must take into accout that the timeout is per DNS lookup so the total DNS wait
+time may be subustantially loner than the value specified in
+.I DNSTimeout
+\.  There is a DNS lookup for each connection if the
+.I InternalHosts
+option is in use and one for DKIM public key record lookup for each algorithm
+per signature per message (i.e. potentially two lookups per signature).
+
+.SH FILES
+.TP
+.I @CONFDIR@/dkimpy-milter.conf
+Default location of this file.
+
+.SH "AUTHORS"
+\ddkimpy-milter\fR was written by Scott Kitterman <scott@kitterman.com>.
+It is based on dkim-milter.py  Copyright (c) 2001-2013 Business Management Systems, Inc.
+Copyright (c) 2013-2015 Stuart D. Gathman
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>.
+.PP
+This man-page was created by Scott Kitterman <scott@kitterman.com>.
+
+.SH COPYRIGHT
+Configuration items derived from OpenDKIM 2.11.0 opendkim.conf.5.in:
+Copyright (c) 2007, 2008, Sendmail, Inc. and its suppliers.  All rights
+reserved.  See LICENSE.Sendmail.
+
+Copyright (c) 2009-2015, The Trusted Domain Project.  All rights reserved.
+See LICENSE.
+
+Updated for dkimpy-milter.  Updates licensed under the same terms as the rest
+of the package.
+Copyright (c) 2018,2019 Scott Kitterman <scott@kitterman.com>

setup.py

@@ -17,24 +17,85 @@
     51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA."""
 
 from setuptools import setup
+import distutils.cmd
+import distutils.log
+import sys
 import os
+import subprocess
 
 description = "Domain Keys Identified Mail (DKIM) signing/verifying milter for Postfix/Sendmail."
 
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+
+class FileMacroExpand(distutils.cmd.Command):
+    description = "Expand @@ variables in input files, simlar to make macros."
+    user_options = [
+        ('sysconfigdir=', 'e', 'Specify system configuration directory. [/usr/local/etc]'),
+        ('sbindir=', 's', 'Specify system binary directory. [/usr/local/sbin]'),
+        ('bindir=', 'b', 'Specify binary directory. [/usr/loca/bin]'),
+        ('rundir=', 'r', 'Specify run state directory. [/run]'),
+    ]
+
+    def initialize_options(self):
+        self.sysconfigdir = '/usr/local/etc'
+        self.sbindir = '/usr/local/sbin'
+        self.bindir = '/usr/local/bin'
+        self.rundir = '/run'
+
+    def finalize_options(self):
+        self.configdir = self.sysconfigdir + '/dkimpy-milter'
+        self.rundir += '/dkimpy-milter'
+
+    def run(self):
+        files = ['etc/dkimpy-milter.conf', 'man/dkimpy-milter.conf.5', \
+                 'system/dkimpy-milter.service', 'system/dkimpy-milter', \
+                 'system/dkimpy-milter.openrc', \
+                 'system/socket-activation/dkimpy-milter.service', \
+                 'system/socket-activation/dkimpy-milter.socket', ]
+        for infile in files:
+            outfile = ''
+            try:
+                filein = open(infile + '.in')
+                for line in filein:
+                    for function in ["@SYSCONFDIR@", "@CONFDIR@", "@SBINDIR@", "@BINDIR@", "@RUNSTATEDIR@"]:
+                        splitline = line.split(function)
+                        if len(splitline) > 1:
+                            if function == "@SYSCONFDIR@":
+                                line = splitline[0] + self.sysconfigdir + splitline[1]
+                            elif function == "@CONFDIR@":
+                                line = splitline[0] + self.configdir + splitline[1]
+                            elif function == "@SBINDIR@":
+                                line = splitline[0] + self.sbindir + splitline[1]
+                            elif function == "@BINDIR@":
+                                line = splitline[0] + self.bindir + splitline[1]
+                            elif function == "@RUNSTATEDIR@":
+                                line = splitline[0] + self.rundir + splitline[1]
+                    outfile += line
+                out = open(infile, 'w')
+                for line in outfile:
+                    out.write(line)
+                out.close()
+            except FileNotFoundError as x:
+                pass
+
 kw = {}  # Work-around for lack of 'or' requires in setuptools.
 try:
     import dns
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'dnspython']
+    kw['install_requires'] = ['dkimpy>=1.1.0', 'pymilter>=1.0.5', 'authres>=1.1.0', 'PyNaCl', 'dnspython>=1.16.0']
 except ImportError:  # If PyDNS is not installed, prefer dnspython
-    kw['install_requires'] = ['dkimpy>=0.7', 'pymilter', 'authres>=1.1.0', 'PyNaCl', 'Py3DNS']
+    kw['install_requires'] = ['dkimpy>=1.1.0', 'pymilter>=1.0.5', 'authres>=1.1.0', 'PyNaCl', 'Py3DNS']
 
 setup(
     name='dkimpy-milter',
-    version='1.1.4',
+    version='1.2.3',
     author='Scott Kitterman',
     author_email='scott@kitterman.com',
     url='https://launchpad.net/dkimpy-milter',
     description=description,
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     download_url = "https://pypi.python.org/pypi/dkimpy-milter",
     classifiers= [
         'Development Status :: 5 - Production/Stable',
@@ -57,10 +118,14 @@ setup(
     include_package_data=True,
     data_files=[(os.path.join('share', 'man', 'man5'),
         ['man/dkimpy-milter.conf.5']), (os.path.join('share', 'man', 'man8'),
-        ['man/dkimpy-milter.8']), ('etc', ['etc/dkimpy-milter.conf']),
+        ['man/dkimpy-milter.8']), (os.path.join('etc', 'dkimpy-milter'),
-        (os.path.join('lib', 'systemd', 'system'),
+        ['etc/dkimpy-milter.conf']), (os.path.join('lib', 'systemd', 'system'),
         ['system/dkimpy-milter.service']),(os.path.join('etc', 'init.d'),
-        ['system/dkimpy-milter'])],
+        ['system/dkimpy-milter']), (os.path.join('etc', 'init.d'),
+        ['system/dkimpy-milter.openrc'])],
     zip_safe = False,
+    cmdclass={
+        'expand': FileMacroExpand,
+    },
     **kw
 )

system/dkimpy-milter

@@ -18,10 +18,8 @@
 # Short-Description: dkimpy-milter
 # Description:       Python DKIM Milter for Sendmail and Postfix
 ### END INIT INFO
-prefix="/usr/local"
+sysconfdir="/usr/local/etc/dkimpy-milter"
-exec_prefix=${prefix}
+bindir="/usr/local/bin"
-sysconfdir="/usr/local/etc"
-bindir="${exec_prefix}/bin/"
 RUNDIR="/run/dkimpy-milter"
 DAEMON=${bindir}/dkimpy-milter
 PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
@@ -35,8 +33,8 @@ test -x $DAEMON || exit 0
 
 # Include dkimpy-python defaults if available
 # Typically not used
-if [ -f /etc/default/dkimpy-milter ] ; then
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
-	. /etc/default/dkimpy-milter
+	. $sysconfdir/default/dkimpy-milter
 fi
 
 set -e
@@ -120,7 +118,7 @@ case "$1" in
         echo "$NAME."
 	;;
   status)
-        status_of_proc -p /var/run/dkimpy-milter/dkimpy-milter.pid /usr/local/bin/dkimpy-milter dkimpy-milter
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
         ;;
 
   *)

system/dkimpy-milter.in

@@ -0,0 +1,131 @@
+#! /bin/sh
+#
+# skeleton	example file to build /etc/init.d/ scripts.
+#		This file should be used to construct scripts for /etc/init.d.
+#
+#		Written by Miquel van Smoorenburg <miquels@cistron.nl>.
+#		Modified for Debian 
+#		by Ian Murdock <imurdock@gnu.ai.mit.edu>.
+#
+# Version:	@(#)skeleton  1.9  26-Feb-2001  miquels@cistron.nl
+#
+### BEGIN INIT INFO
+# Provides:          dkim-milter dkim-milter-python dkimpy-milter
+# Required-Start:    $remote_fs $syslog $network $time
+# Required-Stop:     $remote_fs $syslog $network
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: dkimpy-milter
+# Description:       Python DKIM Milter for Sendmail and Postfix
+### END INIT INFO
+sysconfdir="@CONFDIR@"
+bindir="@BINDIR@"
+RUNDIR="@RUNSTATEDIR@"
+DAEMON=${bindir}/dkimpy-milter
+PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:
+NAME=dkimpy-milter
+DESC="Python DKIM Milter"
+USER=dkimpy-milter
+GROUP=dkimpy-milter
+SOCKET=$RUNDIR/dkimpy-milter.sock
+
+test -x $DAEMON || exit 0
+
+# Include dkimpy-python defaults if available
+# Typically not used
+if [ -f $sysconfdir/default/dkimpy-milter ] ; then
+	. $sysconfdir/default/dkimpy-milter
+fi
+
+set -e
+
+. /lib/lsb/init-functions
+
+case "$1" in
+  start)
+	echo -n "Starting $DESC: "
+	# Create the run directory if it doesn't exist
+	if [ ! -d $RUNDIR ]; then
+		install -o $USER -g $GROUP -m 755 -d $RUNDIR || return 2
+	fi
+
+	# Clean up stale sockets
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		pid=`cat $RUNDIR/$NAME.pid`
+		if ! ps -C $DAEMON -s $pid >/dev/null; then
+			rm $RUNDIR/$NAME.pid
+			# UNIX sockets may be specified with or without the
+			# local: prefix; handle both
+			t=`echo $SOCKET | cut -d: -f1`
+			s=`echo $SOCKET | cut -d: -f2`
+			if [ -e $s -a -S $s ]; then
+				if [ "$t" = "$s" -o "$t" = "local" ]; then
+					rm $s
+				fi
+			fi
+		fi
+	fi
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+	echo "$NAME."
+	;;
+  stop)
+	echo -n "Stopping $DESC: "
+	if [ -f $RUNDIR/$NAME.pid ]; then
+		chown root:root $RUNDIR/$NAME.pid
+		start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid 
+		rm $RUNDIR/$NAME.pid
+		#echo $SOCKET
+		if [ -e $SOCKET ]; then
+			rm $SOCKET
+		fi
+	fi
+	echo "$NAME."
+	;;
+  force-reload)
+        echo -n "Force reloading $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        sleep 1
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+        ;;
+  restart)
+        echo "Restarting $DESC: "
+        echo -n "Stopping $DESC: "
+        if [ -f $RUNDIR/$NAME.pid ]; then
+                chown root:root $RUNDIR/$NAME.pid
+                start-stop-daemon --stop --pidfile $RUNDIR/$NAME.pid
+                rm $RUNDIR/$NAME.pid
+                #echo $SOCKET
+                if [ -e $SOCKET ]; then
+                        rm $SOCKET
+                fi
+        fi
+        echo "$NAME."
+	sleep 1
+        echo -n "Starting $DESC: "
+        start-stop-daemon --start --background --quiet --pidfile \
+                $RUNDIR/$NAME.pid --exec $DAEMON $sysconfdir/$NAME.conf
+        echo "$NAME."
+	;;
+  status)
+        status_of_proc -p $RUNDIR/$NAME.pid $DAEMON dkimpy-milter
+        ;;
+
+  *)
+	N=/etc/init.d/$NAME
+	echo "Usage: $N {start|stop|force-reload|restart|}" >&2
+	exit 1
+	;;
+esac
+
+exit 0

system/dkimpy-milter.openrc

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="/usr/local/etc/dkimpy-milter/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="/usr/local/bin/dkimpy-milter"
+pidfile="/run/dkimpy-milter/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.openrc.in

@@ -0,0 +1,15 @@
+#!/sbin/openrc-run
+# Copyright 1999-2019 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+CONFFILE="@CONFDIR@/${RC_SVCNAME}.conf"
+required_files="${CONFFILE}"
+
+command="@BINDIR@/dkimpy-milter"
+pidfile="@RUNSTATEDIR@/${RC_SVCNAME}.pid"
+command_args="${CONFFILE} -P ${pidfile}"
+
+depend() {
+	use dns logger net
+	before mta
+}

system/dkimpy-milter.service

@@ -6,7 +6,7 @@ After=network.target
 [Service]
 Type=simple
 PIDFile=/run/dkimpy-milter/dkimpy-milter.pid
-ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter.conf 
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf -P /run/dkimpy-milter/dkimpy-milter.pid
 
 [Install]
 WantedBy=multi-user.target

system/dkimpy-milter.service.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5) 
+After=network.target
+
+[Service]
+Type=simple
+PIDFile=@RUNSTATEDIR@/dkimpy-milter.pid
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf -P @RUNSTATEDIR@/dkimpy-milter.pid
+
+[Install]
+WantedBy=multi-user.target

system/socket-activation/dkimpy-milter.service

@@ -4,7 +4,7 @@ Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
 Requires=dkimpy-milter.socket
 
 [Service]
-ExecStart=/usr/bin/dkimpy-milter /etc/dkimpy-milter.conf
+ExecStart=/usr/local/bin/dkimpy-milter /usr/local/etc/dkimpy-milter/dkimpy-milter.conf
 User=dkimpy-milter
 
 [Install]

system/socket-activation/dkimpy-milter.service.in

@@ -0,0 +1,11 @@
+[Unit]
+Description=DKIMpy Milter
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+Requires=dkimpy-milter.socket
+
+[Service]
+ExecStart=@BINDIR@/dkimpy-milter @CONFDIR@/dkimpy-milter.conf
+User=dkimpy-milter
+
+[Install]
+Also=dkimpy-milter.socket

system/socket-activation/dkimpy-milter.socket.in

@@ -0,0 +1,12 @@
+[Unit]
+Description=DKIMpy Milter socket
+Documentation=man:dkimpy-milter(8) man:dkimpy-milter.conf(5)
+
+[Socket]
+ListenStream=@RUNSTATEDIR@/dkimpy-milter.sock
+SocketMode=0660
+# override SocketGroup to grant access to members of another system group:
+SocketGroup=dkimpy-milter
+
+[Install]
+WantedBy=sockets.target

tests/02_sign_message.miltertest

@@ -26,7 +26,7 @@ function connect_and_send (sockname, headers, body)
    end
 
    -- mt.macro(conn, SMFIC_MAIL, "i", "simple-message")
-   if mt.mailfrom(conn, "<alice@example.net>") ~= nil then
+   if mt.mailfrom(conn, "<alicüe@example.net> (Alicþþÿÿe)") ~= nil then
       error "mt.mailfrom() failed"
    end
    if mt.getreply(conn) ~= SMFIR_CONTINUE then
@@ -69,7 +69,7 @@ function connect_and_send (sockname, headers, body)
    return conn
 end
 
-for _, keytype in ipairs({"ed25519", "rsa"}) do
+for _, keytype in ipairs({"ed25519", "rsa", "ed25519.stable", "rsa.stable", "ed25519.table", "rsa.table"}) do
    mt.echo("testing "..keytype)
    signing = connect_and_send("unix:"..keytype..".signing.sock", msg.headers, msg.body)
    -- verify that a test header field got added

tests/runtests

@@ -15,10 +15,19 @@ for keytype in "${KEY_TYPES[@]}"; do
     if [ "$keytype" = ed25519 ]; then
         keyfile=KeyFileEd25519
         selector=SelectorEd25519
-    else
+    elif [ "$keytype" = rsa ]; then
         keyfile=KeyFile
         selector=Selector
     fi
+    if [ "$keytype" = ed25519 ]; then
+        keytable=KeyTableEd25519
+        signingtable=SigningTable
+        selector=SelectorEd25519
+    elif [ "$keytype" = rsa ]; then
+        keytable=KeyTable
+        signingtable=SigningTable
+        selector=Selector
+    fi
     cat > "$keytype.signing.conf" <<EOF
 Domain example.net
 $keyfile testkey.$keytype.key
@@ -34,7 +43,55 @@ Socket unix:$keytype.verify.sock
 PidFile $keytype.verify.pid
 Mode v
 DNSOverride $(cat testkey.$keytype.dns)
+MinimumKeyBits 2048
 UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.conf" <<EOF
+$keyfile testkey.$keytype.key
+$selector testkey
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.stable.signing.sock
+PidFile $keytype.stable.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.stable.verify.conf" <<EOF
+Socket unix:$keytype.stable.verify.sock
+PidFile $keytype.stable.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat > "$keytype.table.conf" <<EOF
+$keytable $WORKDIR/$keytype-table
+$signingtable $WORKDIR/signing-table
+Socket unix:$keytype.table.signing.sock
+PidFile $keytype.table.pid
+Mode s
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+cat > "$keytype.table.verify.conf" <<EOF
+Socket unix:$keytype.table.verify.sock
+PidFile $keytype.table.verify.pid
+Mode v
+DNSOverride $(cat testkey.$keytype.dns)
+UserID $(id --name --user):$(id --name --group)
+EOF
+
+    cat >  "$keytype-table" <<EOF
+preskey example.org:testkey:$WORKDIR/testkey.$keytype.key
+orgkey  example.org:testkey:$WORKDIR/testkey.$keytype.key
+netkey  example.net:testkey:$WORKDIR/testkey.$keytype.key
+EOF
+
+    cat > "signing-table" <<EOF
+president@example.org	@special.example.org:preskey
+*@example.org		orgkey
+*@example.net		netkey
 EOF
 done
 
@@ -42,7 +99,7 @@ cleanup() {
     echo cleaning up jobs:
     jobs
     for keytype in "${KEY_TYPES[@]}"; do
-        for func in signing verify; do
+        for func in signing verify stable stable.verify table table.verify; do
             if [ -s "$keytype.$func.pid" ] && kill -0 "$(cat "$keytype.$func.pid")"; then
                 kill "$(cat $keytype.$func.pid)"
             fi
@@ -50,7 +107,7 @@ cleanup() {
     done
     wait
     for keytype in "${KEY_TYPES[@]}"; do
-        for func in signing verify; do
+        for func in signing verify stable stable.verify table table.verify; do
             errdata="$keytype.$func.stderr"
             if [ -s "$errdata" ]; then
                 printf -- "-> %s:\n" "$errdata"
@@ -63,7 +120,7 @@ cleanup() {
 }
 
 for keytype in "${KEY_TYPES[@]}"; do
-    for func in signing verify; do
+    for func in signing verify stable stable.verify table table.verify; do
         PYTHONPATH="$(dirname "$TESTDIR")" "$DKIMPY_MILTER" "$keytype.$func.conf" 2>"$keytype.$func.stderr" &
     done
 done
@@ -82,3 +139,5 @@ for x in ${TESTS:-"$TESTDIR"/*.miltertest}; do
     printf -- "-> running %s...\n" "$x"
     miltertest -s "$x"
 done
+
+rm -rf "$(dirname $TESTDIR)/dkimpy_milter/__pycache__"