tahina.priv.at/ spam

ich / me

vorträge / talks

spam

hacks

electronics

impressum

cm@tahina.priv.at

  1. anti-spam measures
  2. Newer stuff (that doesn't really work)
  3. Selective Greylisting
    1. Details
    2. Implementation
    3. Documentation
    4. Revision history
    5. Download
    6. Bugs

anti-spam measures

I've been hardening my mail server against spam for a long time, and I've built a SpamAssassin-based antispam appliance for customers. The basics of anti-spam configuration can be found in my talks on the subject.

Newer stuff (that doesn't really work)

I've also experimented with greylisting and SPF. The problem with both is that they interfere too much with the delivery of legitimate mail -- "standard" greylisting is applied to all mail, and therefore delays delivery of legitimate messages, while SPF has all kinds of well-known problems with forwarding, people that send mails from all over the world, and spammers creating SPF records for their domains.

Selective Greylisting

So in the end, after the umpteenth usenet discussion ("greylisting is evil because it clogs my servers queues" vs "greylisting is so effective"), I implemented a postfix policy daemon for "Selective Greylisting".

Basically, it means that messages that look OK are immediately whitelisted, while messages that look "suspect" in one way or another are subject to greylisting.

Interestingly enough, it seems to work quite well, and that's why I'm creating this page.

(I'll try to extract some statistics from the logfiles and add some numbers to this page RSN)

Details

Selective Greylisting, as "classic" greylisting, takes place at the RCPT stage in the SMTP dialog. A list of checks is done, and a "suspectness" score is generated as the count of the failed tests. If the suspectness is zero, the combination of client IP address, HELO string and MAIL FROM domain is whitelisted, a non-zero suspectness results in greylisting (for a period of time depending on the score [I'm not sure this detail has any beneficial effect]).

The checks that I've implemented are:

  • Reverse DNS lookup

    I never liked the idea of blocking clients with no reverse lookup, but in this context it seems acceptable.

  • HELO name is an IP address or non-FQDN

    This, again, is a test that on its own doesn't say much about a host's legitimacy.

  • HELO name exists in DNS

    Here, I test whether the HELO name has an A or MX record.

  • Client is on a dialup-DNSBL

    This is again something on which other people have been blocking mail for ages, which I considered too harsh.

  • SPF status is either "pass" or "none"

    This seems like a good use of SPF to me: if everything is fine, fine. If not, greylist. I'm using the "guess" mode of Mail::SPF::Query here, which, in the absence of SPF DNS records, tries to guess them from the sender domain's DNS data.

The tuple that is grey- or whitelisted is (client IP address, HELO name, sender domain) -- this is in contrast to "standard" greylisting, where (client IP address, sender address, recipient address) is used. I think that my approach has a few benefits:

  • It doesn't impose a penalty on hosts sending mail from or to many users -- after all, mail hosts are usually associated with domains and not users.

  • It keeps the database smaller.

  • By including the HELO string, it is effective against spam senders which often use randomly selected HELOs (In fact, during the Dec. 05 Sober.U worm run, I've seen clients try to send with as many as 20 different HELO strings).

Last, but not least, an important feature: I'm keeping a manually- maintained list of forwarders, i.e. machines where an email address is being forwarded to my mail server. Machines on that list are automatically whitelisted, so they don't have to go thru the greylist cycle for every new sender domain that happens to be forwarded to me, plus they're immune from DNS misconfigurations etc.

Implementation

My implementation of the concept is a policy delegation daemon for postfix. Postfix' smtpd sends all available data to the daemon when the RCPT command is received.

Add the following to /etc/postfix/master.cf:

selective-greylist     unix -  n       n       -       -       spawn
        user=nobody argv=/path/to/selective-greylist -D

Add this somewhere appropriate in smtpd_recipient_restrictions in main.cf:

check_policy_service unix:private/selective-greylist

And create the directory for the database; /var/spool/postfix/db/greylist/ is what is defined in the source. This directory must be writeable by the "nobody" account.

Install the prerequisite Perl modules:

  • Mail::SPF::Query
  • Net::DNS
  • DBI
  • DBD::SQLite

Don't forget to fill the "forwarders" table when the sqlite DB has been created:

$ sqlite /var/spool/postfix/db/greylist/greylist.sqlite
SQLite version 2.8.15
Enter ".help" for instructions
sqlite> insert into forwarders values('forwarder1.example.com');

Documentation

Here's the man page: selective-greylist.man.html.

Revision history

Version 1.1

  • -m was broken
  • problems with DBD::SQLite vs DBD::SQLite2 fixed

Version 1.0

  • renamed program to selective-greylist
  • flock(2) the sqlite database file, because sqlite's locking is lacking; tested on a rather busy mailserver
  • integrated expiry of whitelisted and greylisted entries
  • integrated a patch from Raphael Wegmann to stop accepting mail based on system load average
  • added a few commandline parameters
  • added POD documentation
  • made it available as a debian package

Version ?

This was the first published version, named meta-greylist, which lacked all the features that are present in the newer versions :-)

Download

The Debian packages should work on sarge.

Bugs

None that I know of.