pymilter/TODO

Don't match dynamic ptr in bestguess.

When content filtering is not installed, reject BLACKLISTed MFROM
immediately.  There is no use waiting until EOM.

Configuration is problematic when handling incoming, but not outgoing mail.
The problem comes when alice@example.com sends mail to bill@example.com,
and we are the MX for example.com, but alice is sending from some other
MTA.  The mail is flagged external, so we don't list example.com in
internal_domains (or we would get "spam from self").  But, if we try to do a
CBV, we get "fraudulent MX", because the MX is ourself!  So we need to 
avoid doing CBV on such domains.  Currently, we try to make sure the SPF
policies don't do CBV.

We now don't check internal domains for incoming mail if there is an
SPF record.

On the other hand, if alice is sending internally, or with SMTP AUTH, she
*does* need the domain to be in internal_domains.  The solution to that 
is to use the new SMTP AUTH access configuration to specify which domains
can be used by smtp AUTH (by user if desired).

It would be cleaner if CBV would know which domains we have agreed to
be MX for. Some ideas for external connections:

a) check access file for To:example.com	RELAY
b) check mailertable
c) check mx_domains config list
d) if there is an SPF record, don't check internal_domains
   (let SPF block unauthorized machines)

But that still doesn't handle the roaming user, who won't use SMTP
AUTH, but sends through some hotel MTA.  Maybe we don't want to support
him?

When setting up pydspam, both sender and rcpt must resolve to dspam users
for falsepositive recognition.  Usually, this means adding 
honeypot@mail.example.com to alias list for honeypot in pymilter.cfg.
This needs to be documented.  I was caught by it setting up a new site.

Add signature (x-sig=AB7485f=TS) to Received-SPF, so it can be used
to blacklist sources of delayed DSNs.

rcpt-addr may let us know when a recipient is unknown.  That should count
against reputation.

Need to use wildcards in blacklist.log: *.madcowsrecord.net
Need to exclude emails like !*-admin@example.com in whitelist_sender.
Need to exclude robot users from autowhitelist.  Don't want to have to
list all users, so implement something like !*-admin@bmsi.com,@bmsi.com.

GOSSiP feedback from user training is ignored because UMIS has already been
removed from queue.  Maybe keep UMIS in queue, and add method to 
alter last feedback for ID.

Generate DSNs according to RFC 3464

Get temperror policy from access file.

Reporting explanation for failure should show source if sender
provided explanation.

Bug in Auto-whitelist.  Recent Auto-whitelist doesn't override expired entry.

SPF permerror diagnostics should include corrected mechanism.

Delay SPF check until RCPT TO.  Cache result to avoid repeating
for multiple RCPT.  This avoids overhead for invalid RCPT, and
allows for per RCPT local policy.

Check SPF for outgoing mail (including local policy for internal addresses).
This could also solve the second part of the mail from relay problem below.

Whitelisted senders from trusted relay get PROBATION.  Need to extracted
SPF result from headers - and in the case of mail internal to relay
(e.g. bmsi.com), supply 'pass' result.  

For selected domains, check rcpts via CBV before accepting mail.  Cache
results.  This will kick out dictonary attacks against a mail domain
behind a gateway sooner.

Add auto-blacklisted senders to blacklist.log with timestamp.
Add emails blacklisted via CBV so that they are remembered across milter
restarts.

Make all dictionaries work like honeypot.  Do not train as ham unless
whitelisted.  Train on blacklisted messages, or spam feedback.  This
can be called Train On Error.  Should be possible to startup
with training on everything to get dictionary built fast, then switch
to train on error to minimize labor.

Allow unsigned DSNs from selected domains (that don't accept signed MFROM,
e.g. verizon.net).

Allow verified hostnames for trusted_relay.  E.g. HELO name that
passes SPF.

Table of sendmail macros for documentation.

When do we get two hello calls?  STARTTLS is one reason.

Option: accept mail from auto-whitelisted senders even with spf-fail,
but do not update dspam.  This can be done for individual senders or domains 
using the access file.

pysrs: SRS doesn't get applied to proper recipients when there are
multiple recipients.  This requires debugging cf scripts - yuk.

auto_whitelist false_positives from quarantine - perhaps only when
user selects special button (use special header to communicate
that from dspamcgi.py to milter.)

Use send_dsn.log for blacklist also.  AddrCache needs localpart
wildcard (e.g. empty localpart).

Quarantined mail is missing headers modified/added by milter after
checking dspam.

Send DSN for permerror before processing extended result.  An additional
DSN may be sent based on extended result.  Send permerror DSN to
postmaster@sending_domain.

Rescind whitelist for banned extensions, in case sender is infected.

Train honeypot on error only.

Find rfc2822 policy for MFROM quoting.

Support explicit errors for SPF policy in access file:
SPF-Neutral:aol.com	ERROR:"550 AOL mail must get SPF PASS"

Defer TEMPERROR in SPF evaluation - give precedence to security
(only defer for PASS mechanisms).

Create null config that does nothing - except maybe add Received-SPF
headers.  Many admins would like to turn features on one at a time.

Can't output messages with malformed rfc822 attachments.

Move milter,Milter,mime,spf modules to pymilter
milter package will have bms.py application

Web admin interface
message log for automated stats and blacklisting
Skip dspam when SPF pass? NO
Report 551 with rcpt on SPF fail?
check spam keywords with character classes, e.g.
	{a}=[a@ãä], {i}=[i1í], {e}=[eë], {o}=[o0ö]

Implement RRS - a backdoor for non-SRS forwarders.  User lists non-SRS 
forwarder accounts, and a util provides a special local alias for the
user to give to the forwarder.  (Or user just adds arbitrary alias
unique to that forwarder to a database.)  Alias only works for mail from that
forwarder.  Milter gets forwarder domain from alias and uses it to
SPF check forwarder.

Framework for modular Python milter components within a single VM.
Python milters can be already be composed through sendmail by running each in
a separate process.  However, a significant amount of memory is wasted
for each additional Python VM, and communication between milters
is cumbersome (e.g., adding mail headers, writing external files).

Copy incoming wiretap mail, even though sendmail alias works perfectly
for the purpose, to avoid having to change two configs for a wiretap.

Provide a way to reload milter.cfg without stopping/restarting milter.

Allow selected Windows extensions for specific domains via milter.cfg

Fix setup.py so that _FFR_QUARANTINE is automatically defined when
available in libmilter.

Keep separate ismodified flag for headers and body.  This is important
when rejecting outgoing mail with viruses removed (so as not to
embarrass yourself), and also removing Received headers with hidepath.

Need a test module to feed sample messages to a milter though a live 
sendmail and SMTP.  The mockup currently used is probably not very accurate,
and doesn't test the threading code.

DONE Require signed MFROM for all incoming bounces when signing all outgoing
mail - except from trusted relays.

DONE Added Message-ID header to DSN with SRS signed sender.  When seen on
incoming rfc ignorant failure message, blacklist sender.

DONE Option to add Received-SPF header, but never reject on SPF.
  I think the above will handle this.

DONE Received-SPF header field should show identity that was checked.

DONE When training with spam, REJECT after data so that mistakenly blacklisted
senders at least get an error.

DONE Milter won't start when it can't change permissions on *.lock to match
*.log.  Should maybe ignore that error - the effect will be to set
the permissions to default.

DONE Milter won't start when a whitelist/blacklist file is missing.

DONE Delayed failure detection should parse From header to find email address.

DONE When bms.py can't find templates, it passes None to dsn.create_msg(),
which uses local variable as backup, which no longer exist.  Do plain
CBV in that case instead.

DONE Find and use X-GOSSiP: header for SPAM: and FP: submissions.  Would need
to keep tags longer.

DONE Parse incoming 3464 DSNs for "Action: failed" to recognize delayed
failures.  This works regardless of Subject.

DONE Reports PROBATION even when rejecting message (works, but confusing in
log).

DONE Delayed_failure detection needs to handle multi-line header fields.  
Also, delayed_failure should be recognized when addressed to
postmaster@helodomain 

DONE DSN for Permerror shows 'None' for error under some condition.

DONE Allow blacklisted emails as well as domains in blacklist.log.  Use same
data structure as autowhitelist.log.  

DONE Backup copies for outgoing/incoming mail.