Fighting Email Spam

30 July 2009

Email spam and email viruses are a problem now for eveyone, but there are some simple measures you can take to limit the damage to your Inbox. First we're going to present some background to the problem and then take a look at what you can do, or what we can do to help.

A Recent History of Spam

It used to be that individuals and companies were using their own servers to distribute millions of emails, but that's no longer a viable model since spam 'blackhole lists' became widespread. A single server sending lots of email is quickly identified and added to one or more of these lists. Mail servers around the world will refuse incoming email from the servers that are black-listed.

Chirp subscribes to six separate blackhole lists to protect our users. Our mail server also performs a number of tests on the sending domain to make sure that it's legitimate. We have a monitoring system in place that detects suspicious email activity and temporarily blocks the server in question at the firewall level. Persistant offenders are either added to the firewall permanently or to our internal blackhole list.

Unfortunately that's no longer enough. In recent years the challenge has evolved and the pattern we now see is emails coming from not one, but thousands of servers. These are Windows computers that have been 'hacked' and turned into so-called 'spam zombies'. That means that the hacker or spammer is able to use other peoples computers to distribute their message with very little risk of being blocked by blackhole lists. The result being that it's now almost impossible to identify spam email based on it's point of origin.

So the battle moved to the client-side with improvements to desktop- and web-mail clients. This involved the introduction of 'Bayesian filters'. These are algorithms that not only filter out spam and viruses, but also have the ability to learn and improve themselves based on user actions. Because most spam email includes common words and formatting these filters were able to detect and block 80-90% of unwanted emails.

Again the spammers have reacted, this time by introducing new, smarter, forms of spam. They're cunningly sending out masses of email containing randomly generated text that's formatted like a normal email. When you flag these messages as spam it confuses your Bayesian filter. Emails that would normally be classified as 'good' or 'safe' are now flagged as containing spam. This reduces the 'confidence' with which the filter can classify emails, meaning that the efficiency of the filter is reduced and a greater proportion of spam will get through.

So what can be done about turning back the tide? You can see from the above that it's become increasingly difficult to block spam based on the source or even the content (apart from obvious attachments such as known viruses) and that's set to continue at least until some new filtering technologies are available.

The answer then is to go back to the start. We're recommending to our clients that they take more steps to protect their email addresses to prevent them falling into the hands of spammers in the first place.

Protecting your Email Address

The absolute golden rule for avoiding spam is to stop spammers getting their hands on your email address, because once that happens it's entered into a database and sold and re-sold around the world. It's estimated that in 2006 there are at least 200 highly organised criminal groups involved in spam, phishing and email fraud.

So how do the spammers go about collecting ('harvesting') emails? Basically they rely on search engines and related programs that scour websites, forums, guestbooks, mailing lists and similar media for email addresses. Some also target random addresses ('info', 'webmaster', 'accounts', ...) at your domain to see if the message is accepted (indicating that they may have found an actual email address) or bounced (indicating a miss).

Chirp has taken a number of measures to prevent this from happening:

1. all email addresses that have been entered into your CMS are automatically encrypted when they appear on your website. They may look and behave like normal email links, but the HTML source code that search engines and spammers read is encrypted using a unique algorithm that they can't easily break;
2. other email addresses that appear on your website need to be encrypted as well. We do this as a matter of course and can provide instructions for clients who update their own HTML on how to do this;
3. previously it was common practice for a domain to have a 'catch-all' email alias, meaning that any address at your domain would be accepted. We've now stopped this practice as it makes it much too easy for spammers. Now each domain will have a list of specific addresses that can receive email and any incoming message addressed to someone else will be rejected;

That's what we're doing, now to what you can do. We can see from our logs which email addresses are receiving the most emails and usually that's because they're being spammed. The first thing we do is to search for that address in Google to see if it shows up in search engine results. If even one page or document is listed that contains your email address then you have a problem.

If you want to keep using an address that already appears in Google then you need to immediately contact the people responsible for the website(s) where it appears and ask them to either remove the page or document or to update it to remove the email address or to replace it with a link to your website. Replacing an email link with a web link has an added benefit of improving your search engine ranking.

If you've posted to a 'news group' or public mailing list that displays email links then it's almost impossible to have them updated. News groups in particular are copied across thousands of different servers around the world. If your address appears there then you'll probably have to have it disabled and switch to using a new one.

Based on this it should be obvious that any information relating to you or your organisation that has any possibility of ending up on the Internet should never contain an email address. This includes PDF, Word and similar files. And be careful about posting your address to any kind of discussion on the web unless you know it's going to be protected. Even on widely distributed print publications you should be careful as more and more 'hard copy' text is being scanned and digitalised.

For those who still want to be able to post to news groups and similar forums then it's advisable to set up one or more disposable addresses that you can use for a while and then retire when the spam becomes a problem.

Conclusion

It should be clear from this discussion that it's becoming increasingly difficult to treat the problem of spam emails and that the only viable solution is prevention, or regular innoculation by switching to new addresses - not helpful if you need to alert all your contacts every time.

If you are having problems with the amount of spam and virus email in your Inbox then please contact us. If you can provide copies of the emails WITH FULL HEADERS then we can help identify the source of the problem and advise on whether you need to change your email address or take other measures.