DandelionNStuff/pymilter
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f357a237b22a21870e576540e4f3a0bbae22cf31
pymilter/milter.html
T
Stuart Gathman 9fb3ad70d4 Development changes since 0.7.2
2005-05-31 18:23:49 +00:00

779 lines
36 KiB
HTML
Raw Blame History

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Python Milters</title>
</head><body>
<P ALIGN="CENTER"><A HREF="http://www.anybrowser.org/campaign/">
<IMG SRC="/art/brain1.gif"
ALT="Viewable With Any Browser" BORDER="0"></A>
<img src="/art/banner_4.gif" width="468" height="60" border="0"
usemap="#banner_4" alt="Your vote?">
<map name="banner_4">
<area shape="rect" coords="330,25,426,59"
href="http://education-survey.org/" alt="I Disagree">
<area shape="rect" coords="234,28,304,57" href="http://www.honestEd.com/" alt="I Agree">
</map>
</P>
<h1 align=center>Sendmail Milters in Python</h1>
<h4 align=center>by <a href="mailto:%75%72%6D%61%6E%65%40%6E%65%75%72%61l%61%63%63%65%73%73%2E%63%6F%6D">Jim Niemira</a>
and <a href="mailto:%73%74%75%61%72%74%40%62%6D%73%69%2E%63%6F%6D">
Stuart D. Gathman</a><br>
This web page is written by Stuart D. Gathman<br>and<br>sponsored by
<a href="http://www.bmsi.com">Business Management Systems, Inc.</a> <br>
Last updated Jan 05, 2005</h4>
See the <a href="faq.html">FAQ</a> | <a href="#download">Download now</a> |
<a href="/mailman/listinfo/pymilter">Subscribe to mailing list</a> |
<a href="#overview">Overview</a>
<p>
<a href="//www.python.org">
<img src="python55.gif" align=left alt="A Python"></a>
<a href="//www.sendmail.org/">Sendmail</a> introduced a
<a href="http://www.milter.org/milter_api/api.html"> new API</a> beginning with version 8.10 -
libmilter. The milter module for <a href="//www.python.org">Python</a>
provides a python interface to libmilter that exploits all its features.
<p>
Sendmail 8.12 officially releases libmilter.
Version 8.12 seems to be more robust, and includes new privilege
separation features to enhance security.
I recommend upgrading.
<h2> Recent Changes </h2>
Release 0.7.2 tightens the authentication screws with a "3 strikes and
your out" policy. A sender must have a valid PTR, HELO, or SPF record
to send email. Specific senders can be whitelisted using the
"delegate" option in the spf configuration section by adding a
default SPF record for them. The PTR and HELO are required
by RFC anyway, so this is not an unreasonable requirement.
There is now a coherent policy for an SPF softfail result. A softfail
is accepted if there is a valid PTR or HELO, or if the domain
is listed in the "accept_softfail" option of the spf configuration section.
A neutral result is accepted by default if there is a valid PTR or
HELO, (and the SPF record was not guessed), unless the domain is listed in the
"reject_neutral" option. Common forms of PTR records for dynamic IPs are
recognized, and do not count as a valid PTR. This does not prevent anyone
from sending mail from a dynamic IP - they just need to configure a
valid HELO name or publish an SPF record.
<p>
As SPF adoption continues to rise, forged spam is not getting through. So
spammers are publishing their SPF records as predicted. The 0.7.2 RPM
now provides the <code>rhsbl</code> sendmail hack so that spammer domains
can be blacklisted. With the RPM installed, add a line like the following
to your <code>sendmail.mc</code>.
<pre>
HACK(rhsbl,`blackholes.example.com',"550 Rejected: " $&{RHS} " has been spamming our customers.")dnl
</pre>
<p>
Of course, spammers are now starting to register
throwaway domains. The next thing we need is a custom DNS server,
in Python, that
can recognize patterns. For instance, one spammer registers ded304.com,
ded305.com, ded306.com, etc. We also need the custom DNS server to
let SPF classic clients check SES (which will be part of pysrs).
The <a href="http://twistedmatrix.com/products/twisted">Twisted Python</a>
framework provides a custom DNS server - but I
would like a smaller implementation for our use.
<p>
The RPM for release 0.7.0 moves the config file and socket locations to
/etc/mail and /var/run/milter respectively. We now parse Microsoft CID records
- but only hotmail.com uses them. They seem to have applied for a patent on
the brilliant idea of examining the mail headers to see who the message is
from. We aren't doing that here, so not to worry - but I am not a lawyer, so
if you are worried, change spf.py around line 626 to return None instead of
calling CIDParser(). There is a new option to reject mail with no PTR
and no SPF.
<p>
Microsoft is pushing an anti-opensource license for their pending patent
along with their sender-ID proposal before the IETF.
It is royalty free - but requires anyone distributing a binary they've
compiled from source to sign a license agreement. The Apache Software
Foundation <a
href="http://www.apache.org/foundation/docs/sender-id-position.html"> explains
the problem with sender-ID</a>, and Debian <a
href="http://www.debian.org/News/2004/20040904">concurs</a>. Since
the <a href="http://download.microsoft.com/download/4/3/9/439b024b-09fd-44ee-8ff0-10e834004c36/senderid_FAQ.PDF">Microsoft license</a> is
<a href="http://www.circleid.com/article/732_0_1_0_C/">incompatible with free
software in general</a> and the <a
href="http://www.imc.org/ietf-mxcomp/mail-archive/msg03678.html">GPL in
particular</a>, Python milter will not be able to implement sender-ID in its
current form. This was, no doubt, Microsoft's intent all along.
<p>
Sender-ID attempts to do for RFC2822 headers what SPF does for RFC2821 headers.
Unlike SPF, it has never been tried, and is encumbered by a stupid patent. I
recommend ignoring it and continuing to implement and improve SPF until a
working and unencumbered proposal for RFC2822 headers surfaces.
<p>
<a href="http://spf.pobox.com">
<img src="SPF.gif" align=left alt="SPF logo"></a>
Release 0.6.6 adds support for <a href="http://spf.pobox.com/">SPF</a>,
a protocol to prevent forging of the envelope from address.
SPF support requires <a href="http://pydns.sourceforge.net/">pydns</a>.
The included spf.py module is an updated version of the original 1.6
version at <a href="http://www.wayforward.net/spf/">wayforward.net</a>.
The updated version tracks the draft RFC and test suite.
<p>
The FAQ addresses <a href="faq.html#spf">how to get started with SPF</a>.
<p>
Release 0.6.1 adds a full milter based dspam application.
<p>
I have selected the <a href="http://www.nuclearelephant.com/projects/dspam/">
dspam bayes filter project</a> and <a href="dspam.html">
packaged it for python</a>.
Release 0.6.0 offers a simple application of dspam I call "header triage",
which rejects messages with spammy headers.
To use header triage, you must have <a href="dspam.html">DSPAM</a> installed,
and select a dictionary that is well moderated by someone who gets
lots of spam. That dictionary can be used to block spam that is
obvious from the headers (e.g. X-Mailer and Subject) before it ties
up any more resources. I have yet to see any false positives from this
approach (check the milter log), but if there are, the sender will
get a REJECT with the message "Your message looks spammy."
<h2> Enough Already! </h2>
Nearly a dozen people have emailed me begging for a feature to copy
outgoing and/or incoming mail to a backup directory by user. Ok, it
looks like this is a most requested feature for 0.5.6. In the meantime,
here are some things to consider:
<ul>
<li> If you want to equivalent of a Bcc added to each message, this
is very easy to do in the python code for bms.py. See below.
<li> If you want to copy to a file in a directory (thus avoiding having to
set up aliases), this is slightly more involved. The bms.py milter already
copies the message to a temporary file for use in replacing the message body
when banned attachments are found. You have to open a file, and copy the
Mesage object to it in eom().
<li> Finally, you are probably aware that most email clients already
keep a copy of outgoing mail? Presumably there is a good reason for
keeping another copy on the server.
</ul>
<p>
To Bcc a message, call <code>self.add_recipient(rcpt)</code> in envfrom after
determining whether you want to copy (e.g. whether the sender is local). For
example,
<pre>
def envfrom(...
...
if len(t) == 2:
self.rejectvirus = t[1] in reject_virus_from
if t[0] in wiretap_users.get(t[1],()):
self.add_recipient(wiretap_dest)
if t[1] == 'mydomain.com':
self.add_recipient('&lt;copy-%s&gt;' % t[0])
...
</pre>
<p>
To make this a generic feature requires thinking about how the configuration
would look. Feel free to make specific suggestions about config file
entries. Be sure to handle both Bcc and file copies, and designating what
mail should be copied. How should "outgoing" be defined? Implementing it is
easy once the configuration is designed.
<h3><a name=overview>Overview</a></h3>
This package provides a robust toolkit for Python <a
href="#milter">milters</a>, and the beginnings of a general purpose mail
filtering system written in Python.
<p>
At the lowest level, the 'milter' module provides a thin wrapper around the
<a href="http://www.milter.org/milter_api/api.html">
sendmail libmilter API</a>. This API lets you register callbacks for
a number of events in the
<a href="http://www.cs.concordia.ca/~group/fig/public/email/relay/milter+ruleset-checks.html">process of sendmail receiving a message via SMTP</a>.
These events include the initial connection from a MTA,
the envelope sender and recipients, the top level mail headers, and
the message body. There are options to mangle all of these components
of the message as it passes through the milter.
<p>
At the next level, the 'Milter' module (note the case difference) provides a
Python friendly object oriented wrapper for the low level API. To use the
Milter module, an application registers a 'factory' to create an object
for each connection from a MTA to sendmail. These connection objects
must provide methods corresponding to the libmilter callback events.
<p>
Each event method returns a code to tell sendmail whether to proceed
with processing the message. This is a big advantage of milters over
other mail filtering systems. Unwanted mail can be stopped in its
tracks at the earliest possible point.
<p>
The Milter.Milter class provides default implementations for event
methods that
do nothing, and also provides wrappers for the libmilter methods to mutate
the message.
<p>
The 'spf' module provides an implementation of <a href="http://spf.pobox.com">
SPF</a> useful for detecting email forgery.
<p>
The 'mime' module provides a wrapper for the Python email package that
fixes some bugs, and simplifies modifying selected parts of a MIME message.
<p>
Finally, the bms.py application is both a sample of how to use the
Milter and spf modules, and the beginnings of a general purpose SPAM filtering,
wiretapping, SPF checking, and Win32 virus protecting milter. It can
make use of the <a href="pysrs.html">pysrs</a> package when available for
SRS/SES checking and the <a href="dspam.html">pydspam</a> package for Bayesian
content filtering. SPF checking
requires <a href="http://pydns.sourceforge.net/">
pydns</a>. Configuration documentation is currently included as comments
in the <a href="milter.cfg">sample config file</a> for the bms.py milter.
<h3><a name=download>Downloading</a></h3>
The latest stable release is <a href="#stable">0.7.2</a>. A stable
release is one which has been installed (and working correctly) on
production systems long enough to convince me that it is stable. As
the package gains more features and complexity, stable will mean no
bug reports from outside users either.
<p>
The latest version is 0.7.2-2. See the <a href=NEWS>Change Log</a>.
PLEASE NOTE - if you are using the modules, but not the bms milter application,
then ignore the RPMs and milter.spec. Use 'python setup.py bdist_rpm' to
build source and binary rpms that do not include the milter application.
<p>
I want to split the bms milter application to a new project once I figure
out the renaming. The current plan is to rename 'milter' to 'pymilter', which
will have the Python modules. The bms milter application will still be named
'milter' and depend on pymilter (so that my installs won't notice anything).
<p>
<a name="stable"><b>Stable</b></a>
<a href="http://bmsi.com/python/milter-0.7.2.tar.gz">
milter-0.7.2.tar.gz</a> Three strikes and your out policy. Some SPF fixes.
Recognizes PTR records for dynamic IPs.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.7.2-2.i386.rpm">
milter-0.7.2-2.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.7.2-2rh9.i386.rpm">
milter-0.7.2-2rh9.i386.rpm</a> Binary RPM for Redhat 9, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.7.2-2.src.rpm">
milter-0.7.2-2.src.rpm</a> Source RPM for Redhat 9,7.x.
<p>
<a href="http://bmsi.com/python/milter-0.7.1.tar.gz">
milter-0.7.1.tar.gz</a> Support setmlreply, handle some more exceptions
for malformed spam. Compiling pymilter with sendmail-8.12.10, requires
sendmail-devel with _FFR_MULTILINE set. The binary will work with older
sendmails. The _FFR_MULTILINE option only affects libmilter.a.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.7.1-1.i386.rpm">
milter-0.7.1-1.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.7.1-1.src.rpm">
milter-0.7.1-1.src.rpm</a> Source RPM for Redhat 9,7.x.
<p>
<a href="http://bmsi.com/python/milter-0.7.0.tar.gz">
milter-0.7.0.tar.gz</a> Move config file and default socket location.
Parse M$ CID records.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.7.0-1.i386.rpm">
milter-0.7.0-1.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.7.0-1rh9.i386.rpm">
milter-0.7.0-1rh9.i386.rpm</a> Binary RPM for Redhat 9, requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/aix/milter-0.7.0-1.ppc.rpm">
milter-0.7.0-1.ppc.rpm</a> Binary RPM for AIX, requires sendmail-8.13.1.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.7.0-1.src.rpm">
milter-0.7.0-1.src.rpm</a> Source RPM for Redhat 9,7.x.
<p>
<a href="http://bmsi.com/python/milter-0.6.9.tar.gz">
milter-0.6.9.tar.gz</a> Add SPF test suite driver, and validate
spf.py against test suite. Add best_guess and get_header to spf.py.
Libmilter timeout option in config.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.6.9-1.i386.rpm">
milter-0.6.9-1.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.6.9-1.src.rpm">
milter-0.6.9-1.src.rpm</a> Source RPM for Redhat 9,7.x.
<p>
<a href="http://bmsi.com/python/milter-0.6.8.tar.gz">
milter-0.6.8.tar.gz</a> Include Received-SPF headers in Dspam analysis.
Fix sysv init for Redhat 9 and later. Reject bounces with multiple
recipients.
<br>
<a href="http://bmsi.com/python/milter-0.6.8.patch">milter-0.6.8.patch</a>
Last minutes fixes from production testing.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.8-3.i386.rpm">
milter-0.6.8-3.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh9/milter-0.6.8-3.src.rpm">
milter-0.6.8-3.src.rpm</a> Source RPM for Redhat 9,7.x.
<p>
<a href="http://bmsi.com/python/milter-0.6.7.tar.gz">
milter-0.6.7.tar.gz</a> Explicit local socket bug,
<a href="http://spf.pobox.com/srs.html">SRS</a> forgery detection,
thread resource starvation detection.
SRS support requires <a href="http://bmsi.com/python/pysrs.html">pysrs</a>.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.7-3.i386.rpm">
milter-0.6.7-3.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.6.7-3.src.rpm">
milter-0.6.7-3.src.rpm</a> Source RPM for Redhat 7.x.
Release 0.6.7-3 patches:
<ul>
<li> Defang message/rfc822 content_type with boundary
<li> Support SPF delegation
<li> Reject neutral SPF result for selected domains
</ul>
<p>
<a href="http://bmsi.com/python/milter-0.6.6.tar.gz">
milter-0.6.6.tar.gz</a> Plug another memory leak,
<a href="http://spf.pobox.com/">SPF</a> support, hello blacklist.
SPF support requires <a href="http://pydns.sourceforge.net/">pydns</a>.
NOTE - the spf.py module included is modified from the official 1.6
version at <a href="http://www.wayforward.net/spf/">wayforward.net</a>.
I neglected to add the CVS log. The changes are expanded result codes
and tolerating common method misspellings in SPF records. I have notified the
author, but haven't heard back. At some point, the RPM will
include the official pyspf tarball and apply patches.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.6-2.i386.rpm">
milter-0.6.6-2.i386.rpm</a> Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and <a href="http://www.python.org/2.3.3/rpms.html">
python2.3</a>. Release 2 fixes sysv init script bug for python2.3.
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.6.6-2.src.rpm">
milter-0.6.6-2.src.rpm</a> Source RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/python/milter-0.6.5.tar.gz">
milter-0.6.5.tar.gz</a> Plug memory leak, progress reporting, trusted relay.
Redhat RPM now requires sendmail-8.12.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.5-2.i386.rpm">
milter-0.6.5-2.i386.rpm</a> Binary RPM for Redhat 7.x
<br>
<a href="http://bmsi.com/linux/rh72/milter-0.6.5-2.src.rpm">
milter-0.6.5-2.src.rpm</a> Source RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/python/milter-0.6.4.tar.gz">
milter-0.6.4.tar.gz</a> Numerous Dspam fixes. Requires
<a href="dspam.html">pydspam-1.1.5</a> and
<a href="/libdspam/dspam.html">dspam-2.6.5.2</a>
for Dspam features. The dspam-python RPM has been replaced by pydspam.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.4-1.i386.rpm">
milter-0.6.4-1.i386.rpm</a> Binary RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/python/milter-0.6.3.1.tar.gz">
milter-0.6.3.1.tar.gz</a> New dspam SCREENER feature with pydspam-1.1.4.
Don't save a defang copy of false positives. Fixed an oops from last fix,
rejecting false positives. BUG: sendmail-8.11 doesn't invoke milter
when sending mail via sendmail from command line (8.12 works). Therefore,
the supplied falsepositive script for milter based dspam doesn't work
with stock RedHat 7.x. I am writing a HOWTO for configuring milter
based dspam that will address this (and a fix in the next version).
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.3-1.i386.rpm">
milter-0.6.3-1.i386.rpm</a> Binary RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/python/milter-0.6.2.tar.gz">
milter-0.6.2.tar.gz</a> work around email.Message.get_filename bug,
dspam_exempt list, REJECT messages with missing MIME boundaries (which
are almost always spam),
DISCARD messages which any dspam user flags as spam,
start.sh was calling python instead of python2 on Linux.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.2-1.src.rpm">
milter-0.6.2-1.src.rpm</a> Source RPM for Redhat 7.x (and likely
higher versions)
<p>
<a href="http://bmsi.com/python/milter-0.6.1.tar.gz">
milter-0.6.1.tar.gz</a> dspam milter application, python-2.2.3 support.
<p>
You must have <a href=dspam.html>dspam and dspam-python</a> loaded for
the dspam feature to work. Brief instructions for configuring are
in the default config file. This is working at a customer, but I'm
sure a few more iterations will be required to make setup as smooth
as possible.
<p>
NOTE: Outlook destroys dspam tags when forwarding mail (while converting
HTML to text). Perhaps some config option will turn this abominable
"feature" off. Working around this by making dspam tags visble on
HTML mail is ugly. My suggestion is to not use Outlook, for this and
many other reasons - especially security. Any other suggestions for
those married to Microsoft are welcome. The DSPAM LDA works around this
by making the tags visible in HTML attachments. This is ugly, and
occasionally corrupts attachments.
<p>
We have to supply workarounds for bugs in the email module (reported
to sourceforge). The workarounds reference some internal variables
which change with python versions.
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.1-1.i386.rpm">
milter-0.6.1-1.i386.rpm</a> Binary RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.1-1.src.rpm">
milter-0.6.1-1.src.rpm</a> Source RPM for Redhat 7.x (and likely
higher versions)
<p>
<a href="http://bmsi.com/python/milter-0.6.0.tar.gz">
milter-0.6.0.tar.gz</a> simple dspam pre-filtering, use email module,
requires python &gt;= 2.2.2.
<ul>
<li> The milter.so module from 0.5.4
is needed to run this release on AIX. Haven't tracked this down yet.
<li> The patches to fix the email packages in mime.py don't work
on python-2.2.3. The email package is still broken in 2.3, and patches
required for that will likely be different still.
</ul>
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.0-1.i386.rpm">
milter-0.6.0-1.i386.rpm</a> Binary RPM for Redhat 7.x
<p>
<a href="http://bmsi.com/linux/rh72/milter-0.6.0-1.src.rpm">
milter-0.6.0-1.src.rpm</a> Source RPM for Redhat 7.x (and likely
higher versions)
<p>
<a href="http://www.bmsi.com/python/milter-0.5.5.tar.gz">
milter-0.5.5.tar.gz</a> IPV6 support, passing None to set_XXX_callback,
set_reply, chg_header, detect internal connections. Note, this release
did not work on AIX4.1.5, probably due to IPV6 support breaking something.
The milter.so module from 0.5.4 can be installed to use this release
with AIX.
<p>
<a href="http://www.bmsi.com/python/milter-0.5.4.tar.gz">
milter-0.5.4.tar.gz</a> wiretap, smart alias features, quarantine support.
<p>
The name of the production "sample" milter "bms.py" now
stands for "Basic Milter System" until someone suggests a better name.
The test coverage is rather
sparse at present.
Please <a href="mailto:%73%74%75%61%72%74%40%62%6D%73%69%2E%63%6F%6D">email</a> with proposals for what
to name the milter application.
<h4>NOTES</h4>
<ul>
<li>
Quarantine support requires that you define _FFR_QUARANTINE
when compiling miltermodule.c. I am not sure how to make setup.py
do that for you iff sendmail was actually compiled with _FFR_QUARANTINE.
<li>
While 0.6.0 will use the new email package in Python-2.2, that
package seems to be buggy in Python-2.2.1. The list example in the docs
doesn't find all MIME parts. Update: Python-2.2.2 has fixed the email
package. It can now parse my test cases.
<li>
Preliminary testing with python-2.2 shows that most things work after
adding <code>self.readahead = ""</code> to <code>mimepart.seek</code>.
Python-2.2 <code>multifile</code> reads one less newline per section than
2.1. I'm not not sure which is correct. After adding some calls to
<code>rstrip()</code> in testmime.py, all milter modules pass unit testing
with python-2.2. Python-2.2 patches have been released since 0.5.3.
<li>
sgmlop-1.1a3 has a memory leak (at least Python milter has a
memory leak when using sgmlop instead of sgmllib). Do not make Python
milter use sgmlop-1.1a2 or a3 in a production
system unless you can restart your milter periodically. The amount
of memory leaked seems roughly proportional to the amount of HTML
parsed.
<li>
There are a number of ways that malformed MIME attachments
can cause a python traceback. Uncaught exceptions cause a 415
error to be returned to sendmail. So far, all the malformed messages
I've investigated have been SPAM - so good riddance. I would prefer,
however, that the mime handling libraries were more precise. Beginning
with 0.5.1, bms.py will save messages that cause a traceback during
scanning in the tempfile directory with a ".fail" extension. This
makes it easier to get samples of mail that causes parsing problems
for incorporation into the unit tests.
</ul>
<p>
<a href="http://www.bmsi.com/python/milter-0.5.2.tar.gz">
milter-0.5.2.tar.gz</a> Fix and unittest another HTML parsing bug.<br>
<a href="http://www.bmsi.com/python/milter-0.5.1.tar.gz">
milter-0.5.1.tar.gz</a> Handle encoded rfc822 attachments.<br>
<a href="http://www.bmsi.com/python/milter-0.5.0.tar.gz">
milter-0.5.0.tar.gz</a> Use a config file so users don't have to
keep syncing with bms.py. <br>
<a href="http://www.bmsi.com/python/milter-0.4.5.tar.gz">
milter-0.4.5.tar.gz</a> Work with sgmlop. Reduce local hacks to config variables.
<p>
Python milter is under GPL. The authors can probably be convinced to
change this to LGPL.
<h3>What is a <a name="milter">milter</a>?</h3>
Milters can run on the same machine as sendmail, or another machine. The
milter can even run with a different operating system or processor than
sendmail.
Sendmail talks to the milter via a local or internet socket.
Sendmail keeps the
milter informed of events as it processes a mail connection. At any
point, the milter can cut the conversation short by telling sendmail
to ACCEPT, REJECT, or DISCARD the message. After receiving a complete
message from sendmail, the milter can again REJECT or DISCARD it, but it
can also ACCEPT it with changes to the headers or body.
<h3> What can you do with a milter? </h3>
<menu>
<li> A milter can DISCARD or REJECT spam based based on algorithms scripted
in python rather than sendmail's cryptic "cf" language.
<li> A milter can alter or remove attachments from mail that are poisonous to
Windows.
<li> A milter can scan for viruses and clean them when detected.
<li> A milter scans outgoing as well as incoming mail.
<li> A milter can add and delete recipients to forward or secretly
copy mail.
<li> For more ideas, check the <a href="//www.milter.org">Milter Web Page</a>.
</menu>
<a href="http://www.milter.org/milter_api/api.html">
Documentation</a> for the C API is provided with sendmail. Miltermodule
provides a thin python wrapper for the C API. Milter.py provides a simple
OO wrapper on top of that.
<p>
The Python milter package includes a sample milter that replaces dangerous
attachments with a warning message, discards mail addressed to
MAILER-DAEMON, and demonstrates several SPAM abatement strategies.
The MimeMessage class to do this used to be based on the
<code>mimetools</code> and <code>multifile</code> standard python packages.
As of milter version 0.6.0, it is based on the email standard
python packages, which were derived from the
<a href="http://sourceforge.net/projects/mimelib">mimelib</a> project.
The MimeMessage class patches several bugs in the email package,
and provides some backward compatibility.
<p>
The "defang" function of the sample milter was inspired by
<a href="http://www.roaringpenguin.com/mimedefang/">MIMEDefang</a>,
a Perl milter with flexible attachment processing options. The latest
version of MIMEDefang uses an apache style process pool to avoid reloading
the Perl interpreter for each message. This makes it fast enough for
production without using Perl threading.
<p>
<a href="http://sourceforge.net/projects/mailchecker">mailchecker</a> is
a Python project to provide flexible attachment processing for mail. I
will be looking at plugging mailchecker into a milter.
<p>
<a href="http://software.libertine.org/tmda/">TMDA</a> is a Python project
to require confirmation the first time someone tries to send to your
mailbox. This would be a nice feature to have in a milter.
<p>
There is also a <a href="http://www.milter.org/">Milter community website</a>
where milter software and gory details of the API are discussed.
<h3> Is a milter written in python efficient? </h3>
The python milter process is multi-threaded and startup cost is incurred
only once. This is much more efficient than some implementations that
start a new interpreter for each connection. Testing in a production
environment did not use a significant percentage of the CPU. Furthermore,
python is easily extended in C for any step requiring expensive CPU
processing.
<p>
For example, the HTML parsing feature to remove scripts from HTML attachments
is rather CPU intensive in pure python. Using the C replacement for sgmllib
greatly speeds things up.
<h3> Goals </h3>
<menu>
<li> Implement RRS - a backdoor for non-SRS forwarders. User lists non-SRS
forwarder accounts (perhaps in <code>~/.forwarders</code>), and a util
provides a special local alias for the user to give to the forwarder.
Alias only works for mail from that forwarder. Milter gets forwarder
domain from alias and uses it to SPF check forwarder. Requires
milter to have read access to <code>~/.forwarders</code> or else
a way for user to submit entries to milter database.
<li> The bms.py milter has too many features. Create a framework where
numerous small feature modules can be plugged together in the
configuration.
<li> Create a pure python substitute for miltermodule and libmilter that
implements the <a
href="http://www.duh.org/cvsweb.cgi/~checkout~/pmilter/doc/milter-protocol.txt?rev=1">
libmilter protocol</a> in python.
<li> Find or write a faster implementation of sgmllib. The
<a href="http://www.effbot.org/zone/sgmlop-index.htm">sgmlop package</a>
is not very compatible with
<a href="http://www.python.org/doc/2.1.3/lib/module-sgmllib.html">
Python-2.1 sgmllib</a>, but it is a start, and is supported in
milter-0.4.5 or later.
<li> Implement all or most of the features of
<a href="http://www.roaringpenguin.com/mimedefang/">MIMEDefang</a>.
<li> Follow the official <a href="http://www.python.org/peps/pep-0008.html">
Python coding standards</a> more closely.
<li> Make unit test code more like other python modules.
</menu>
<h3> Confirmed Installations </h3>
Please <a href="mailto:%73%74%75%61%72%74%40%62%6D%73%69%2E%63%6F%6D">email</a>
me if you successfully install milter on a system not mentioned below.
<p>
<table>
<tr>
<th>Operating System</th> <th>Compiler</th> <th>Python</th> <th>Sendmail</th>
<th>milter</th>
<tr>
<td>Mandrake 8.0</td><td>gcc-3.0.1</td><td>2.1.1</td><td>8.12.0</td>
<td>0.3.3</td><tr>
<td>Mandrake 8.0</td><td>gcc-2.96</td><td>2.0</td><td>8.11.2</td>
<td>0.3.6</td><tr>
<td>RedHat 6.2</td><td>egcs-1.1.2</td><td>2.2.2</td><td>8.11.6</td>
<td>0.5.4</td><tr>
<td>RedHat 7.1</td><td>gcc-2.96</td><td>?</td><td>8.12.1</td>
<td>0.3.5</td><tr>
<td>RedHat 7.3</td><td>gcc-2.96</td><td>2.2.2</td><td>8.11.6</td>
<td>0.5.5</td><tr>
<td>RedHat 7.3</td><td>gcc-2.96</td><td>2.3.3</td><td>8.13.1</td>
<td>0.7.2</td><tr>
<td>RedHat 8.0</td><td>gcc-3.2</td><td>2.2.1</td><td>8.12.6</td>
<td>0.5.2</td><tr>
<td>Debian Linux</td><td>gcc-2.95.2</td><td>2.1.1</td><td>8.12.0</td>
<td>0.3.7</td><tr>
<td>Debian Linux</td><td>gcc-3.2.2</td><td>2.2.2</td><td>8.12.7</td>
<td>0.5.4</td><tr>
<td>AIX-4.1.5</td><td>gcc-2.95.2</td><td>2.1.1</td><td>8.11.5</td>
<td>0.3.3</td><tr>
<td>AIX-4.1.5</td><td>gcc-2.95.2</td><td>2.1.1</td><td>8.12.1</td>
<td>0.3.4</td><tr>
<td>AIX-4.1.5</td><td>gcc-2.95.2</td><td>2.1.3</td><td>8.12.3</td>
<td>0.4.2</td><tr>
<td>AIX-4.1.5</td><td>gcc-2.95.2</td><td>2.2.3</td><td>8.13.1</td>
<td>0.7.1</td><tr>
<td>Slackware 7.1</td><td>?</td><td>?</td><td>8.12.1</td>
<td>0.3.8</td><tr>
<td>Slackware 9.0</td><td>gcc-3.2.2</td><td>2.2.3</td><td>8.12.9</td>
<td>0.5.4</td><tr>
<td>OpenBSD</td><td>?</td><td>2.3.3?</td><td>8.13.1?</td>
<td>0.7.2</td><tr>
<td>SuSE 7.3</td><td>gcc-2.95.3</td><td>2.1.1</td><td>8.12.2</td>
<td>0.3.9</td><tr>
<td>FreeBSD</td><td>gcc-2.95.3</td><td>2.2.1</td><td>8.12.3</td>
<td>0.4.0</td><tr>
<td>FreeBSD</td><td>gcc-2.95.3</td><td>2.2.2</td><td>?</td>
<td>0.5.5</td><tr>
<td>FreeBSD 4.4</td><td>gcc-2.95.3</td><td>?</td><td>8.12.10</td>
<td>0.6.6</td><tr>
</table>
<h3> Requirements </h3>
<menu>
<li> While the miltermodule will work with python 1.5, you probably
want to use python 2.0 or better. The python code uses a number of
python 2 features.
<li> Python must be configured with thread support. This is because
sendmail's libmilter requires thread support.
<li> You must compile sendmail with libmilter enabled. In versions of
sendmail prior to 8.12 libmilter is marked FFR (For Future Release) and
is not installed by default.
Sendmail 8.12 still does not enable libmilter by default. You must
explicitly select the "MILTER" option when compiling.
<li> Python milter has been tested against sendmail-8.11 and sendmail-8.12.
<li> Python milter must be compiled for the specific version of sendmail
it will run with. (Since the result is dynamically loaded, there could
conceivably be multiple versions available and selected at startup - but
that will have to wait.) This situation may only exist for sendmail
versions prior to 8.12. The protocol seems designed for backward
compatibility - and 8.12 is the first official milter release.
<li> Mea Culpa! After reading the Python Style guide, I realize that
my Python code is not up to snuff. Apparently mixed tabs and spaces
are anathema to those using Windows editors, where tabs can be expanded using
any arbitrary algorithm. Other than that, my
intuition matched Guido's pretty well - although I like to indent by 2
rather than 4. I will arrange to have tabs expanded to spaces when
exporting new versions. Until then, beware!
</menu>
<h3> <a name="aix4"> AIX 4.1.5 Requirements </a> </h3>
To create sendmail RPMs for AIX, you can download my AIX 4.1.5 spec files
for <a href="/aix/sendmail.spec">sendmail-8.11.5</a>
or <a href="/aix/sendmail12.spec">sendmail-8.12.3</a>. If you have
not already set it up, I use a <a href="/aix/aix.spec">dummy RPM package</a>
to represent the stuff that comes with AIX. You might also want
my <a href="/aix/python.spec">python-2.1.1</a> spec file for AIX. It
does not include Tk or curses modules, sorry. If y'all trust me, you can
download rpms for AIX 4.x from my <a href="/aix">AIX RPM directory</a>.
<p>
Sendmail-8.12 renames
libsmutil.a to libsm.a. Unfortunately, libsm.a is an important AIX system
shared library. Therefore, I rename libsm.a back to libsmutil.a for
AIX. This presents a problem for setup.py.
<h3> <a name="rh72"> RedHat 7.2 Requirements </a> </h3>
If you are running Redhat 7.2, the distributed version of sendmail
now enables libmilter by default. RedHat 7.2 bundles
the development libraries with the main sendmail package, so
there is no sendmail-devel package. However, they forgot to include the
headers! So you'll have to get the SRPM and modify it. I suggest
moving the static libs to a devel package and adding the headers. If
this is too much trouble, you can get the <a href="mfapi.h">mfapi.h</a>
header for sendmail-8.6.11 from here and manually install it as
<code>/usr/include/libmilter/mfapi.h</code>.
<p>
If you do modify the SRPM, I suggest renaming libsmutil.a
to libsm.a - just like sendmail-8.12 will. If you manually install
mfapi.h or don't rename libsmutil.a, you'll
need to force <code>libs = ["milter", "smutil"]</code> in setup.py.
<p>
If you have installed python2, and want
python-milter to use python2, add <code>python=python2</code> to setup.cfg
and build with <code>python2 setup.py bdist_rpm</code>.
<h3> <a name="rh62"> Redhat 6.2 Requirements </a> </h3>
If you are running Redhat 6.2, the distributed version of sendmail
does not enable libmilter. You can download the Redhat 7.2 sendmail.spec
modified to compile on RedHat 6.2:
<a href="http://www.bmsi.com/linux/rh62/sendmail-rhmilter.spec">
sendmail-rhmilter.spec</a>. The <a
href="ftp://updates.redhat.com/7.0/en/os/SRPMS/sendmail-8.11.6-1.7.0.src.rpm">
SRPM for sendmail-8.11.6</a> is available from
<a href="http://www.redhat.com">Redhat</a> under
<a href="http://www.redhat.com/support/errata/RHSA-2001-106.html">
Errata for RH6.2</a>. But that doesn't include the latest security
patches since RH6.2 is no longer supported.
<p>
If y'all trust me, you can pick up source and binary sendmail RPMs for RH6.2
from my <a href="http://www.bmsi.com/linux/rh62">linux downloads</a> directory.
The lastest RPMs were built by taking a RH7.2 SRPMS and removing some
RPM features from the spec file that RH6.2 doesn't support, then
recompiling on RH6.2. You can check this by installing the RH7.2 SRPM,
then diffing my sendmail.spec with theirs. Then run
"rpm -bb sendmail-rhmilter.spec" when you are satisfied.
<p>
If you have installed python2, and want
python-milter to use python2, add <code>python=python2</code> to setup.cfg
and build with <code>python2 setup.py bdist_rpm</code>.
You'll need to install the sendmail-devel package to compile milter.
<hr>
<p>
<a href="http://validator.w3.org/check/referer">
<img border=0 src="/vh32.png" alt=" [ Valid HTML 3.2! ] " height=31 width=88></a>
<a href="http://www.redhat.com">
<img src="/art/powered_by.gif" width="88" height="31" alt=" [ Powered By Red Hat Linux ] " border="0"></a>
</p>
</body></html>
Reference in New Issue View Git Blame Copy Permalink