DandelionNStuff/pymilter
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Release 0.8.3

Browse Source
This commit is contained in:
Stuart Gathman
2005-10-12 16:36:30 +00:00
parent ace3e13685
commit ea82d6d608
8 changed files with 276 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HOWTO
+21 -6
View File
@@ -93,23 +93,38 @@ string blocking.
The sendmail access file, or another readonly database with that The sendmail access file, or another readonly database with that
format, can be used for detail spf policy. SPF access policy format, can be used for detail spf policy. SPF access policy
record are tagged with "SPF-{Result}:". Results are record are tagged with "SPF-{Result}:". Results are
Pass, Neutral, Softfail, Fail, TempError, PermError. Currently supported Pass, Neutral, Softfail, Fail, PermError. Currently supported
policy keywords are OK, CBV, REJECT, TEMPFAIL, ERROR:"550 description". policy keywords are OK, CBV, REJECT. Currently, TempError always
results in TEMPFAIL.
The default policies are as follows: The default policies are set in pymilter.cfg. The defaults
if none of the config options are set are as follows:
SPF-Fail: REJECT SPF-Fail: REJECT
SPF-Softfail: CBV SPF-Softfail: CBV
SPF-Neutral: OK SPF-Neutral: OK
SPF-PermError: REJECT SPF-PermError: REJECT
SPF-TempError: TEMPFAIL
SPF-Pass: OK SPF-Pass: OK
The tag may be followed by a specific domain. For instance, to The tag may be followed by a specific domain. For instance, to
require a Pass from aol.com: require a Pass from aol.com:
SPF-Neutral:aol.com ERROR:"550 AOL mail must get SPF PASS" SPF-Neutral:aol.com REJECT
SPF-Softfail:aol.com ERROR:"550 AOL mail must get SPF PASS" SPF-Softfail:aol.com REJECT
The CBV policy requires a valid HELO name. If the EHLO name is
RFC2822 compliant, then a DSN is sent to the alleged sender. The
template for the DSN is selected according to the SPF result:
Fail: softfail.txt
SoftFail: softfail.txt
Neutral: neutral.txt
PermError: permerror.txt
None: strike3.txt
Pass: strike3.txt
The pass template doesn't make any sense - I assumed that CBV would
never be used with a Pass result.
To be continued. To be continued.
MANIFEST.in
+1
View File
@@ -24,3 +24,4 @@ include milter.rc7
include milter.cfg include milter.cfg
include rhsbl.m4 include rhsbl.m4
include *.txt include *.txt
include *.html
NEWS
+3
View File
@@ -3,9 +3,12 @@ Here is a history of user visible changes to Python milter.
0.8.3 Keep screened honeypot mail, but optionally discard honeypot only mail. 0.8.3 Keep screened honeypot mail, but optionally discard honeypot only mail.
spf_accept_fail option for braindead SPF senders spf_accept_fail option for braindead SPF senders
(treats fail like softfail) (treats fail like softfail)
Option to set SPF policy via sendmail access map.
Option to supply Sender header from MAIL FROM when missing.
Consider SMTP AUTH connections internal. Consider SMTP AUTH connections internal.
Send DSN for SPF errors corrected by extended processing. Send DSN for SPF errors corrected by extended processing.
Send DSN before SCREENED mail is quarantined Send DSN before SCREENED mail is quarantined
Use logging package to keep log lines atomic.
0.8.2 Strict processing limits per SPF RFC 0.8.2 Strict processing limits per SPF RFC
Fixed several parsing bugs under RFC Fixed several parsing bugs under RFC
Support official IANA SPF record (type99) Support official IANA SPF record (type99)
bms.py
+5 -2
View File
@@ -1,6 +1,9 @@
#!/usr/bin/env python #!/usr/bin/env python
# A simple milter that has grown quite a bit. # A simple milter that has grown quite a bit.
# $Log$ # $Log$
# Revision 1.29 2005/10/11 22:50:07 customdesigned
# Always check HELO except for SPF pass, temperror.
#
# Revision 1.28 2005/10/10 23:50:20 customdesigned # Revision 1.28 2005/10/10 23:50:20 customdesigned
# Use logging module to make logging threadsafe (avoid splitting log lines) # Use logging module to make logging threadsafe (avoid splitting log lines)
# #
@@ -551,9 +554,9 @@ def parse_addr(t):
>>> parse_addr('user@example.com') >>> parse_addr('user@example.com')
['user', 'example.com'] ['user', 'example.com']
>>> parse_addr('"user@example.com"') >>> parse_addr('"user@example.com"')
['"user@example.com"'] ['user@example.com']
>>> parse_addr('"user@bar"@example.com') >>> parse_addr('"user@bar"@example.com')
['"user@bar"','example.com'] ['user@bar', 'example.com']
>>> parse_addr('foo') >>> parse_addr('foo')
['foo'] ['foo']
""" """
milter.cfg
+3 -3
View File
@@ -89,11 +89,11 @@ reject_spoofed = 0
;reject_noptr = 0 ;reject_noptr = 0
# always accept softfail from these domains, or send DSN otherwise # always accept softfail from these domains, or send DSN otherwise
;accept_softfail = bounces.amazon.com ;accept_softfail = bounces.amazon.com
# treat fail from these domains like softfail: because their SPF record # Treat fail from these domains like softfail: because their SPF record
# or an important sender is screwed up. Must have valid HELO, however. # or an important sender is screwed up. Must have valid HELO, however.
;accept_fail = custhelp.com ;accept_fail = custhelp.com
# use sendmail access file or similar format for detailed spf policy # Use sendmail access map or similar format for detailed spf policy.
# This will override any defaults set above # SPF entries in the access map will override any defaults set above.
;access_file = /etc/mail/access.db ;access_file = /etc/mail/access.db
# Add MAIL FROM as Sender when Sender is missing and From domain # Add MAIL FROM as Sender when Sender is missing and From domain
# doesn't match MAIL FROM. Outlook and other email clients will then display # doesn't match MAIL FROM. Outlook and other email clients will then display
milter.spec
+4 -1
View File
@@ -146,7 +146,7 @@ rm -rf $RPM_BUILD_ROOT
%files -f INSTALLED_FILES %files -f INSTALLED_FILES
%defattr(-,root,root) %defattr(-,root,root)
%doc README NEWS TODO CREDITS sample.py %doc README HOWTO NEWS TODO CREDITS sample.py
/etc/logrotate.d/milter /etc/logrotate.d/milter
/etc/cron.daily/milter /etc/cron.daily/milter
%ifos aix4.1 %ifos aix4.1
@@ -174,6 +174,9 @@ rm -rf $RPM_BUILD_ROOT
- Consider SMTP AUTH connections internal. - Consider SMTP AUTH connections internal.
- Send DSN for SPF errors corrected by extended processing. - Send DSN for SPF errors corrected by extended processing.
- Send DSN before SCREENED mail is quarantined - Send DSN before SCREENED mail is quarantined
- Option to set SPF policy via sendmail access map.
- Option to supply Sender header from MAIL FROM when missing.
- Use logging package to keep log lines atomic.
* Fri Jul 15 2005 Stuart Gathman <stuart@bmsi.com> 0.8.2-4 * Fri Jul 15 2005 Stuart Gathman <stuart@bmsi.com> 0.8.2-4
- Limit each CNAME chain independently like PTR and MX - Limit each CNAME chain independently like PTR and MX
* Fri Jul 15 2005 Stuart Gathman <stuart@bmsi.com> 0.8.2-3 * Fri Jul 15 2005 Stuart Gathman <stuart@bmsi.com> 0.8.2-3
policy.html
+237
View File
@@ -0,0 +1,237 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<title>Python Milter Mail Policy </title>
</head><body>
<h1> Python Milter Mail Policy </h1>
<h3> Classify connection </h3>
When the SMTP client connects, the connection IP address is
saved for later verification, and the connection
is classified as INTERNAL or EXTERNAL by matching the ip
address against the <code>internal_connect</code> configuration.
IP addresses with no PTR, and PTR names that look like
the kind assigned to dynamic IPs (as determined by a heuristic
algorithm) are flagged as DYNAMIC. IPs that match the
<code>trusted_relay</code> configuration are flagged as TRUSTED.
<p>
Examples from the log file (<i>not</i> the SMTP error message returned):
<pre>
2005Jul29 13:56:53 [71207] connect from p50863492.dip0.t-ipconnect.de at ('80.134.52.146', 1858) EXTERNAL DYN
2005Jul29 18:10:15 [74511] connect from foopub at ('1.2.3.4', 46513) EXTERNAL TRUSTED
2005Jul29 14:41:00 [71805] connect from foobar at ('192.168.0.1', 41205) INTERNAL
2005Jul29 14:41:15 [71806] connect from cncln.online.ln.cn at ('218.25.240.137', 35992) EXTERNAL
</pre>
<p>
Certain obviously evil PTR names are blocked at this point:
"localhost" (when IP is not 127.*) and ".".
<pre>
2005Jul29 14:49:50 [71918] connect from localhost at ('221.132.0.6', 50507) EXTERNAL
2005Jul29 14:49:50 [71918] REJECT: PTR is localhost
</pre>
<h3> HELO Check </h3>
The HELO name provided by the client is saved for later verification
(for example by SPF). We could validate the HELO at this point
by verifying that an A record for the HELO name matches the connect ip.
However, currently we only block certain obvious problems.
HELO names that look like an IP4 address
and ones that match the <code>hello_blacklist</code> configuration
are immediately rejected. The hello_blacklist typically contains
the current MTAs own HELO name or email domains.
Clients that attempt to skip HELO are immediately rejected.
<pre>
2005Jul29 18:10:15 [74512] hello from example.com
2005Jul29 18:10:15 [74512] REJECT: spam from self: example.com
2005Jul29 18:17:09 [74581] hello from 80.191.244.69
2005Jul29 18:17:09 [74581] REJECT: numeric hello name: 80.191.244.69
</pre>
<h3> MAIL FROM Check </h3>
Before calling our milter, sendmail checks a DNS blacklist to
block banned sender domains. We never see a blocked domain.
<p>
The MAIL FROM address is saved for possible use by the smart-alias
feature. First, the <code>internal_domains</code> is used for
a simple screening if defined. If the MAIL FROM for an INTERNAL connection
is NOT in <code>internal_domains</code>, then it is rejected (the
PC is most likely infected and attempting to send out spam).
If the MAIL FROM for an EXTERNAL connection IS in
<code>internal_domains</code>, then the message is immediately rejected.
This is quick and effective for most small company MTAs. For more
complex mail networks, it is too simplistic, and should not be defined.
SPF will handle the complex cases.
<h4> wiretap </h4>
The wiretap feature can screen and/or monitor mail to/from certain
users. If the MAIL FROM is being wiretapped, the recipients are
altered accordingly.
<h4> SPF check </h4>
Finally, the MAIL FROM, connect IP, and HELO name are checked against
any SPF records published via DNS for the alleged sender (MAIL FROM).
If there is no SPF record, we check for a local substitute under the
domain defined in the <code>[spf]delegate</code> configuration.
Further checks depend on the result.
<table border=1>
<tr><th>NONE</th><td>
If there is no SPF record (official or delegated), then we
initiate a "three strikes and your out" regime, which looks for
<b>some</b> form of validated identification.
<ol>
<li>We try a "best guess" SPF record of "v=spf1 a/24 mx/24 ptr". If this
passes, good.
<li> We try to validate the HELO name. First check for an SPF record.
Otherwise, check whether the connect IP matches any A record for
the HELO name, or any A record for any MX name for the HELO name,
or is at least in the same /24 subnet as any of the above.
(In other words, a HELO SPF "best guess" of "v=spf1 a/24 mx/24".)
If so, good. We consider the HELO validated. If the HELO SPF
check fails, we reject the email.
</ol>
<pre>
2005Jul30 19:45:16 [93991] connect from [221.200.41.54] at ('221.200.41.54', 3581) EXTERNAL DYN
2005Jul30 19:45:18 [93991] hello from adelphia.net
2005Jul30 19:45:19 [93991] mail from <wendy.stubbsua@link-it.com> ()
2005Jul30 19:45:19 [93991] REJECT: hello SPF: fail 550 access denied
</pre>
<ol>
<li> If there is a validated PTR name, and it doesn't look
like a dynamic name, good. We consider the connection validated.
</ol>
If any of the above can be validated, we continue on.
If none of the above can be validated, and the <code>[SPF]reject_noptr</code>
option is true, we reject the message immediately with the explanation
that we need some form of valid identification before we accept an email.
If <code>[SPF]reject_noptr</code> is false, we flag the message as
needing Call Back Validation.
The Call Back Valildation sends a DSN to the purported sender informing
them of the lack of identification. If the message is legitimate, the
sender needs to know that their email setup is broken and should be corrected.
If the message is forged, the sender is informed of the forgery,
and their need to publish an SPF record or at least use a valid HELO name.
If the purported sender does not accept the DSN,
then the message is rejected. The CBV status is cached to avoid
annoying the purported sender with too many DSNs. Currently, the DSN
is repeated to the same sender once per month.
<p>
In this example, although 3com.com has no SPF record, we assume that
any legitimate mail from them will at least have a valid HELO or PTR.
<pre>
2005Jul30 23:52:03 [96777] connect from [222.252.233.200] at ('222.252.233.200', 29934) EXTERNAL DYN
2005Jul30 23:52:03 [96777] hello from 3mail.3com.com
2005Jul30 23:52:04 [96777] mail from <etec_nic_family@3mail.3com.com> ()
2005Jul30 23:52:04 [96777] REJECT: no PTR, HELO or SPF
</pre>
</td></tr>
<tr><th>PASS</th><td>
A pass result normally lets the email continue on, but the domain is
tracked for reputation (and may be blocked), and may skip content scanning if
it matches a whitelist.
<pre>
2005Jul24 17:44:26 [2104] mail from <gnucash-devel-bounces@gnucash.org> ('SIZE=4410',)
2005Jul24 17:44:26 [2104] Received-SPF: pass (mail.bmsi.com: domain of gnucash.org
designates 204.107.200.65 as permitted sender)
client-ip=204.107.200.65; envelope-from=gnucash-devel-bounces@gnucash.org; helo=cvs.gnucash.org;
</pre>
</td></tr>
<tr><th>NEUTRAL</th><td>
A neutral result normally lets the email continue on, but the domain is not
tracked for reputation or matched against any whitelists.
Highly forged domains listed in <code>[SPF]reject_neutral</code> are
rejected.
<pre>
2005Jul24 17:41:37 [2070] connect from cp500627-a.dbsch1.nb.home.nl at ('84.27.225.3', 3465) EXTERNAL
2005Jul24 17:41:37 [2070] hello from cp500627-a.dbsch1.nb.home.nl
2005Jul24 17:41:38 [2070] mail from <nwarjejkw@yahoo.com> ()
2005Jul24 17:41:38 [2070] REJECT: SPF neutral for nwarjejkw@yahoo.com
</pre>
</td></tr>
<tr><th>SOFTFAIL</th><td>
A softfail result normally lets the email continue on, but the domain is not
tracked for reputation or matched against any whitelists. Furthermore,
the message is flagged as needing Call Back Validation,
and the highly forged domains listed in <code>[SPF]reject_neutral</code> are
rejected as well.
<p>
At present, we also require a valid HELO or PTR to avoid rejecting
a softfail. But this should probably change to only require a
successful CBV.
<p>
The Call Back Valildation sends a DSN to the purported sender informing
them of the softfail. If the message is legitimate, the sender needs
to know about the softfail so that their email setup can be corrected.
If the message is forged, the sender is informed of the forgery, confirming
that SPF is protecting their reputation and encouraging a rapid transition
to a strict policy. If the purported sender does not accept the DSN,
then the message is rejected. The CBV status is cached to avoid
annoying the purported sender with too many DSNs. Currently, the DSN
is repeated to the same sender once per month.
<pre>
2005Jul24 15:41:33 [801] mail from <Aitp@horafeliz.com> ()
2005Jul24 15:41:33 [801] Received-SPF: softfail (mail.bmsi.com: transitioning domain of horafeliz.com
does not designate 221.184.83.185 as permitted sender)
client-ip=221.184.83.185; envelope-from=Aitp@horafeliz.com;
helo=p8185-ipad30funabasi.chiba.ocn.ne.jp;
2005Jul24 15:41:33 [801] rcpt to <david@example.com> ()
2005Jul24 15:41:35 [801] Subject: Microsoft, Adobe, Macromedia, Corel software. Up to 80% discount.
2005Jul24 15:41:35 [801] X-Mailer: Microsoft Outlook, Build 10.0.2605
2005Jul24 15:41:35 [801] CBV: Aitp@horafeliz.com
2005Jul24 15:41:38 [801] REJECT: CBV: 550 <Aitp@horafeliz.com>: User unknown
</pre>
</td></tr>
<tr><th>FAIL</th><td>
The message is rejected with a reference the SPF why page.
<pre>
2005Jul30 19:53:27 [94070] connect from [212.70.52.16] at ('212.70.52.16', 3192) EXTERNAL DYN
2005Jul30 19:53:27 [94070] hello from winzip.com
2005Jul30 19:53:27 [94070] mail from <dan@winzip.com> ()
2005Jul30 19:53:27 [94070] REJECT: SPF fail 550 SPF fail:
see http://openspf.com/why.html?sender=dan@winzip.com&ip=212.70.52.16
</pre>
</td></tr>
<tr><th>PERMERROR</th><td>
Permanent errors were called "unknown", and are still show that way
in the log. The message is rejected. Previously, we enabled "lax" parsing
of the SPF record, but rejecting is better because it informs the
sender about their problem. The next milter version will
look for a local substitute SPF record (as for a missing SPF record)
before rejecting. This will inform the sender of their problem, but
also let the receiver install a temporary workaround.
<pre>
2005Jul24 18:05:37 [2312] mail from <b-mihdbcgaacaa-becibijh-000-@msg.euxiphipops.com> ()
2005Jul24 18:05:37 [2312] REJECT: SPF unknown 550 SPF Permanent Error:
include mechanism missing domain: include
</pre>
The SPF record for msg.euxiphipops.com looked like this at the time of the
above error:
<pre>
msg.euxiphipops.com TXT "v=spf1 mx ptr a include"
</pre>
</td></tr>
<tr><th>TEMPERROR</th><td>
Temporary errors result in a 451 "Try again later" response. The sender
should retry the message at a later time.
<pre>
2005Jul24 07:33:13 [29846] mail from <quickenloans@rate.quicken.com> ('SIZE=73775', 'BODY=8BITMIME')
2005Jul24 07:33:43 [29846] TEMPFAIL: SPF error 450 SPF Temporary Error: DNS Timeout
</pre>
</td></tr>
</table>
</body>
</html>
testbms.py
+2 -1
View File
@@ -300,4 +300,5 @@ if __name__ == '__main__':
fp = milter._body fp = milter._body
sys.stdout.write(fp.getvalue()) sys.stdout.write(fp.getvalue())
else: else:
unittest.main() #unittest.main()
unittest.TextTestRunner().run(suite())