Email and Spam Information

The email server at newton.cx has a two-step anti-spam system in place:

• First, all incoming email is checked to see if it is coming from a real email address. (This is called a “sender callout check”.) That is, if a mail is sent to someone at newton.cx from the address abc@example.com, then the example.com mailserver is contacted to see if the user abc really exists.

  This check is extremely effective at blocking most spam, but it can block legitimate mail from poorly-configured servers. In particular, some automated messages (payment confirmations, newsletters, etc.) are sent from email addresses that do not accept mail. Such messages will initially be rejected by newton.cx. Once a week, the list of rejected messages is reviewed, and any legitimate-looking email addresses are added to a filter allowing them to be delivered. So these messages eventually get through, but may be delayed up to a week. If you are expecting an important automated email from a website, contact postmaster@newton.cx so that the filter can be configured more quickly.

• Secondly, newton.cx is equipped with an adaptive anti-spam system called DSpam. This software is also very good at catching any spam that passes the initial checks, but you must “train” it for it to be of any use. See below for more information. Once again, if you do not train DSpam, it will not help you. Fortunately, training is easy and it usually only takes DSpam a week or two to get very, very good at catching spam.

Training the DSpam Filter

DSpam is trained by putting messages in special mail folders stored on the newton.cx server. Because the POP3 protocol downloads mail off the server, it cannot be used to train DSpam. DSpam must be trained either by using an email reader with the IMAP protocol, or by using the newton.cx webmail interface. Both of these options will show you your server-stored mail folders and allow you to move messages between them.

If you use one of the above options, you should see a folder called DSpamFilters in your mail account. Within that folder, there are several subfolders:

• The FalseNegatives folder is the most important. If you receive a spam message, move it into this folder. Once an hour, DSpam will process this folder and “learn” from this message to improve its spam filtering.
• The NewlyArrivedSpam and NewlyArrivedBorderline folders are where DSpam puts emails that it thinks are spam. The former is for emails that seem very likely to be spam and the latter is for messages that might be legitimate. (This breakdown is based on the mathematical models used by DSpam, and errs on the side of putting messages into NewlyArrivedBorderline.) Once you are happy with DSpam's filtering, you can pretty much ignore NewlyArrivedSpam, but you should check it periodically for legitimate messages that were incorrectly identified as spam. It's recommended that you monitor NewlyArrivedBorderline to make sure the spam filter isn't getting overenthusiastic.
• If you find any such messages in either NewlyArrived folder, you should copy them into the FalsePositives folder. DSpam will process this folder once an hour as well and “learn” from the messages to avoid misclassifying emails like them in the future. You should also copy the message out of the FalsePositives folder into one more appropriate for the email.
• Spam messages more than fourteen days old are moved into the SpamArchive folder. There is no need to open this folder unless you want to read your old spam.
• Innocent messages that were incorrectly identified as spam are moved into the InnocentArchive folder after being processed from the FalsePositives folder. In case you accidentally delete the original message, you can recover it from here.

If you have any questions about the spam filtering or believe it is making too many mistakes, email postmaster@newton.cx.

Technical Details

The SMTP server used by newton.cx is Exim. Spam filtering is provided by Exim's sender callout filtering and DSpam. The DSpam training is done by a highly customized version of the DSpam Mail Trainer (DMT). Mail can be filtered by users with procmail before being delivered into Maildir mailboxes. Access via IMAP and POP3 are provided by Dovecot. Webmail access is provided by SquirrelMail. I recommend all of these programs enthusiastically.