Split off milter applications.
This commit is contained in:
Stuart Gathman
@@ -1,251 +1,3 @@
|
||||
The recent feature to let a REJECT policy for SPF None be overridden
|
||||
by whitelisting is working for CSI and CMS. However, there could be
|
||||
a sender that we want to REJECT even when whitelisted - because they
|
||||
normally get a guessed PASS. Need another policy name - or else just
|
||||
add them to local SPF so they won't ever get 'None'.
|
||||
|
||||
When policy is OK, do not use cbv_cache for blacklist.
|
||||
|
||||
Add postmaster option or general rcpt list to dsn. Can send dsn to
|
||||
user and postmaster on the same connection.
|
||||
|
||||
Check ESMTP NOTIFY before sending real DSNs. Just use CBV if DSNs are
|
||||
not wanted.
|
||||
|
||||
Support CBV to local domains and cache results so that invalid users
|
||||
can be rejected without maintaining valid user lists.
|
||||
|
||||
Now that we blacklist IPs for too many bad rcpts, delay SPF until RCPT TO.
|
||||
|
||||
When content filtering is not installed, reject BLACKLISTed MFROM
|
||||
immediately. There is no use waiting until EOM.
|
||||
|
||||
Configuration is problematic when handling incoming, but not outgoing mail.
|
||||
The problem comes when alice@example.com sends mail to bill@example.com,
|
||||
and we are the MX for example.com, but alice is sending from some other
|
||||
MTA. The mail is flagged external, so we don't list example.com in
|
||||
internal_domains (or we would get "spam from self"). But, if we try to do a
|
||||
CBV, we get "fraudulent MX", because the MX is ourself! So we need to
|
||||
avoid doing CBV on such domains. Currently, we try to make sure the SPF
|
||||
policies don't do CBV. The real solution is for users to use SMTP AUTH,
|
||||
but some of them are stubborn.
|
||||
|
||||
We now don't check internal domains for incoming mail if there is an
|
||||
SPF record.
|
||||
|
||||
On the other hand, if alice is sending internally, or with SMTP AUTH, she
|
||||
*does* need the domain to be in internal_domains. The solution to that
|
||||
is to use the new SMTP AUTH access configuration to specify which domains
|
||||
can be used by smtp AUTH (by user if desired).
|
||||
|
||||
It would be cleaner if CBV would know which domains we have agreed to
|
||||
be MX for. Some ideas for external connections:
|
||||
|
||||
a) check access file for To:example.com RELAY
|
||||
b) check mailertable
|
||||
c) check mx_domains config list
|
||||
d) if there is an SPF record, don't check internal_domains
|
||||
(let SPF block unauthorized machines)
|
||||
|
||||
But that still doesn't handle the roaming user, who won't use SMTP
|
||||
AUTH, but sends through some hotel MTA. Maybe we don't want to support
|
||||
him?
|
||||
|
||||
When setting up pydspam, both sender and rcpt must resolve to dspam users
|
||||
for falsepositive recognition. Usually, this means adding
|
||||
honeypot@mail.example.com to alias list for honeypot in pymilter.cfg.
|
||||
This needs to be documented. I was caught by it setting up a new site.
|
||||
|
||||
Add signature (x-sig=AB7485f=TS) to Received-SPF, so it can be used
|
||||
to blacklist sources of delayed DSNs.
|
||||
|
||||
rcpt-addr may let us know when a recipient is unknown. That should count
|
||||
against reputation.
|
||||
|
||||
Need to use wildcards in blacklist.log: *.madcowsrecord.net
|
||||
Need to exclude emails like !*-admin@example.com in whitelist_sender.
|
||||
Need to exclude robot users from autowhitelist. Don't want to have to
|
||||
list all users, so implement something like !*-admin@bmsi.com,@bmsi.com.
|
||||
|
||||
GOSSiP feedback from user training is ignored because UMIS has already been
|
||||
removed from queue. Maybe keep UMIS in queue, and add method to
|
||||
alter last feedback for ID.
|
||||
|
||||
Generate DSNs according to RFC 3464
|
||||
|
||||
Get temperror policy from access file.
|
||||
|
||||
Reporting explanation for failure should show source if sender
|
||||
provided explanation.
|
||||
|
||||
Bug in Auto-whitelist. Recent Auto-whitelist doesn't override expired entry.
|
||||
|
||||
SPF permerror diagnostics should include corrected mechanism.
|
||||
|
||||
Delay SPF check until RCPT TO. Cache result to avoid repeating
|
||||
for multiple RCPT. This avoids overhead for invalid RCPT, and
|
||||
allows for per RCPT local policy.
|
||||
|
||||
Check SPF for outgoing mail (including local policy for internal addresses).
|
||||
This could also solve the second part of the mail from relay problem below.
|
||||
|
||||
Whitelisted senders from trusted relay get PROBATION. Need to extracted
|
||||
SPF result from headers - and in the case of mail internal to relay
|
||||
(e.g. bmsi.com), supply 'pass' result.
|
||||
|
||||
Add auto-blacklisted senders to blacklist.log with timestamp.
|
||||
Add emails blacklisted via CBV so that they are remembered across milter
|
||||
restarts.
|
||||
|
||||
Make all dictionaries work like honeypot. Do not train as ham unless
|
||||
whitelisted. Train on blacklisted messages, or spam feedback. This
|
||||
can be called Train On Error. Should be possible to startup
|
||||
with training on everything to get dictionary built fast, then switch
|
||||
to train on error to minimize labor.
|
||||
|
||||
Allow unsigned DSNs from selected domains (that don't accept signed MFROM,
|
||||
e.g. verizon.net).
|
||||
|
||||
Allow verified hostnames for trusted_relay. E.g. HELO name that
|
||||
passes SPF.
|
||||
|
||||
When do we get two hello calls? STARTTLS is one reason.
|
||||
|
||||
Option: accept mail from auto-whitelisted senders even with spf-fail,
|
||||
but do not update dspam. This can be done for individual senders or domains
|
||||
using the access file.
|
||||
|
||||
pysrs: SRS doesn't get applied to proper recipients when there are
|
||||
multiple recipients. This requires debugging cf scripts - yuk.
|
||||
|
||||
auto_whitelist false_positives from quarantine - perhaps only when
|
||||
user selects special button (use special header to communicate
|
||||
that from dspamcgi.py to milter.)
|
||||
|
||||
Use send_dsn.log for blacklist also. AddrCache needs localpart
|
||||
wildcard (e.g. empty localpart).
|
||||
|
||||
Quarantined mail is missing headers modified/added by milter after
|
||||
checking dspam.
|
||||
|
||||
Send DSN for permerror before processing extended result. An additional
|
||||
DSN may be sent based on extended result. Send permerror DSN to
|
||||
postmaster@sending_domain.
|
||||
|
||||
Rescind whitelist for banned extensions, in case sender is infected.
|
||||
|
||||
Train honeypot on error only.
|
||||
|
||||
Find rfc2822 policy for MFROM quoting.
|
||||
|
||||
Support explicit errors for SPF policy in access file:
|
||||
SPF-Neutral:aol.com ERROR:"550 AOL mail must get SPF PASS"
|
||||
|
||||
Defer TEMPERROR in SPF evaluation - give precedence to security
|
||||
(only defer for PASS mechanisms).
|
||||
|
||||
Create null config that does nothing - except maybe add Received-SPF
|
||||
headers. Many admins would like to turn features on one at a time.
|
||||
|
||||
Can't output messages with malformed rfc822 attachments.
|
||||
|
||||
Move milter,Milter,mime,spf modules to pymilter
|
||||
milter package will have bms.py application
|
||||
|
||||
Web admin interface
|
||||
message log for automated stats and blacklisting
|
||||
Skip dspam when SPF pass? NO
|
||||
Report 551 with rcpt on SPF fail?
|
||||
check spam keywords with character classes, e.g.
|
||||
{a}=[a@ãä], {i}=[i1í], {e}=[eë], {o}=[o0ö]
|
||||
|
||||
Implement RRS - a backdoor for non-SRS forwarders. User lists non-SRS
|
||||
forwarder accounts, and a util provides a special local alias for the
|
||||
user to give to the forwarder. (Or user just adds arbitrary alias
|
||||
unique to that forwarder to a database.) Alias only works for mail from that
|
||||
forwarder. Milter gets forwarder domain from alias and uses it to
|
||||
SPF check forwarder.
|
||||
|
||||
Framework for modular Python milter components within a single VM.
|
||||
Python milters can be already be composed through sendmail by running each in
|
||||
a separate process. However, a significant amount of memory is wasted
|
||||
for each additional Python VM, and communication between milters
|
||||
is cumbersome (e.g., adding mail headers, writing external files).
|
||||
|
||||
Copy incoming wiretap mail, even though sendmail alias works perfectly
|
||||
for the purpose, to avoid having to change two configs for a wiretap.
|
||||
|
||||
Provide a way to reload milter.cfg without stopping/restarting milter.
|
||||
|
||||
Allow selected Windows extensions for specific domains via milter.cfg
|
||||
|
||||
Fix setup.py so that _FFR_QUARANTINE is automatically defined when
|
||||
available in libmilter.
|
||||
|
||||
Keep separate ismodified flag for headers and body. This is important
|
||||
when rejecting outgoing mail with viruses removed (so as not to
|
||||
embarrass yourself), and also removing Received headers with hidepath.
|
||||
|
||||
Need a test module to feed sample messages to a milter though a live
|
||||
sendmail and SMTP. The mockup currently used is probably not very accurate,
|
||||
and doesn't test the threading code.
|
||||
|
||||
DONE Table of sendmail macros for documentation. In API docs on milter.org.
|
||||
|
||||
DONE For selected domains, check rcpts via CBV before accepting mail. Cache
|
||||
results. This will kick out dictonary attacks against a mail domain
|
||||
behind a gateway sooner.
|
||||
|
||||
DONE Convert DSN to REJECT unless sender gets SPF pass or best guess pass. Make
|
||||
configurable by SPF result with NOTSPAM policy (reject or deliver without DSN).
|
||||
Maybe policy should be NODSN - still verify sender with CBV.
|
||||
|
||||
DONE Add parseaddr test case for 'foo@bar.com <baz@barf.biz>'
|
||||
|
||||
DONE Require signed MFROM for all incoming bounces when signing all outgoing
|
||||
mail - except from trusted relays.
|
||||
|
||||
DONE Added Message-ID header to DSN with SRS signed sender. When seen on
|
||||
incoming rfc ignorant failure message, blacklist sender.
|
||||
|
||||
DONE Option to add Received-SPF header, but never reject on SPF.
|
||||
I think the above will handle this.
|
||||
|
||||
DONE Received-SPF header field should show identity that was checked.
|
||||
|
||||
DONE When training with spam, REJECT after data so that mistakenly blacklisted
|
||||
senders at least get an error.
|
||||
|
||||
DONE Milter won't start when it can't change permissions on *.lock to match
|
||||
*.log. Should maybe ignore that error - the effect will be to set
|
||||
the permissions to default.
|
||||
|
||||
DONE Milter won't start when a whitelist/blacklist file is missing.
|
||||
|
||||
DONE Delayed failure detection should parse From header to find email address.
|
||||
|
||||
DONE When bms.py can't find templates, it passes None to dsn.create_msg(),
|
||||
which uses local variable as backup, which no longer exist. Do plain
|
||||
CBV in that case instead.
|
||||
|
||||
DONE Find and use X-GOSSiP: header for SPAM: and FP: submissions. Would need
|
||||
to keep tags longer.
|
||||
|
||||
DONE Parse incoming 3464 DSNs for "Action: failed" to recognize delayed
|
||||
failures. This works regardless of Subject.
|
||||
|
||||
DONE Reports PROBATION even when rejecting message (works, but confusing in
|
||||
log).
|
||||
|
||||
DONE Delayed_failure detection needs to handle multi-line header fields.
|
||||
Also, delayed_failure should be recognized when addressed to
|
||||
postmaster@helodomain
|
||||
|
||||
DONE DSN for Permerror shows 'None' for error under some condition.
|
||||
|
||||
DONE Allow blacklisted emails as well as domains in blacklist.log. Use same
|
||||
data structure as autowhitelist.log.
|
||||
|
||||
DONE Backup copies for outgoing/incoming mail.
|
||||
|
||||
DONE Don't match dynamic ptr in bestguess.
|
||||
Support smfi_negotiate and auto negotiate only those callbacks for which
|
||||
Milter.Milter methods have been overridden. (Python should be able to
|
||||
do that.)
|
||||
|
||||
Reference in New Issue
Block a user