Nobody likes receiving spam, yet it accounts for over 50% of all email. As such, spam filtering has become big business. We use a combination of RBL, firewall rules and heuristics to help prevent spam in your inbox. We’ve recently been further developing our email delivery systems to ensure your sent mail arrives at its destination.
Historically, each server would deal with its own email. This seems like a good idea as spam from one server won’t affect another, so problems are less likely. As each server sends a relatively small amount of mail, the trust needed to enable successful delivery to big providers like hotmail, gmail etc was proving difficult to build. Hotmail, for example, doesn’t save reputation data for any host sending less than 100 emails to them per day, resulting in an instant ban if they suspect an email is spam.
At the other end of the spectrum, if we run a single outgoing email server and a user sends spam or has their account compromised, we run the risk of mail for all users being blocked. So, a new solution was needed.
Our new email delivery system
Gradually, over a number of months, we built up the reputation of a range of IP addresses for sending mail. Once these were prepared, we’ve set up a system using these IPs for mail servers. All outgoing mail from any of our servers (once checked for mail volume limits and spam/virus content) is forwarded to this new service for delivery.
Mail delivery is attempted from a random address, successful for around 95% of mail. Where it cannot deliver it passes the mail back into the queue to retry. This may be due to rate limiting, reputation, filter rules or any other reason at the receiver end. The delivery is re-attempted from a different IP address until it is successful. Where a receiving server indicates that an IP has been blacklisted, this is temporarily removed from the pool for investigation.
System Information
We use some excellent open-source software to power our mail systems. As with all our services, CentOS is our Linux of choice. Beyond that, the mail queue is stored in a MongoDB database, a Redis instance is used to store session data and cache DNS lookups and ZoneMTA, powered by NodeJS is used to deliver mail.
