In Part 1, we gave an overview of how to maintain a good reputation with email service providers (ESPs) and how to host your email infrastructure if you want to manage email deliverability yourself. Part 2 discussed how IP addresses, sending volume, DNS and authentication affect deliverability. In Part 3, we looked at how to maintain email lists, bounce handling and spam complaints. And Part 4 covered how to handle unsubscribes when they happen, recipient engagement and whitelisting. In this fifth and final installment, we’ll take a look at how to write good emails that don’t trigger spam filters.

There are a few tricks to remember about content besides the mantra “sending something people want.” One of the things Mailgun offers our customers is the ability to set up a test mailbox at Mailgun and enable our spam filters to receive a “spamicity” score to test how your content is judged by spam filters. Absent that, here are a few tips to consider.

• Personalize your emails to each recipient. Ideally, the content should reflect recipients’ specific interests or usage patterns in your application. At least address them by their name…don’t be rude! Mailgun has recipient variables that you can define and use with your email templates to achieve detailed levels of personalization.
• It is best to send multi-part emails using both text and HTML, or text only. Sending HTML only email is not well received by email service providers (ESPs). Also, remember that ESPs generally block images by default so HTML only will not look very good unless users are proactive about enabling images. There are a few tools available to test how your email will render across ESPs and browsers. Litmus offers one, as does Return Path.
• The higher the text to link and text to image ratios, the better. Too many links and images trigger spam flags at ESPs.
• Misspellings, spammy words (buy now!, Free!) are big spam flags, as are ALL CAPS AND EXCLAMATION MARKS!!!!!!!!!!!!!
• The domains in the from field, return-path and message-id should match the domain you are sending from.
• Make sure you are using unsubscribe links and headers in your emails. Many ESPs (particularly Hotmail) pay attention to this and if they are not there, you are likely to get filtered. You can always use Mailgun’s auto unsubscribe handling if you don’t want to deal with this on your end.
• Gmail pays particularly close attention to Message ID and Received headers. Message IDs that are formed incorrectly (without brackets <> and with wrong domain after @) can make Gmail think you are a spammer. The simplest way to create the right Message ID with Mailgun is to not include one. Then Mailgun will create a perfect Message ID for you.
• Links should include the domain that is sending the email. Also, popular URL shorteners can be a bad idea because they are frequently used by spammers
• A/B test your emails to optimize recipient engagement. Subject lines are particularly important. You can use Mailgun’s tagging and tracking statistics to measure A/B testing and improve your content.

So that’s it. We hope that this Email Best Practices Deliverability guide will help you send better emails and make sure they are delivered — no matter which provider you use. Happy Sending!

In Part 1, we gave an overview of how to maintain a good reputation with email service providers (ESPs) and how to host your email infrastructure if you want to manage email deliverability yourself. Part 2 discussed how IP addresses, sending volume, DNS and Authentication affect deliverability. In Part 3, we looked at how to maintain email lists, bounce handling and spam complaints. Today, we’ll discuss how to handle unsubscribes when they happen, recipient engagement and whitelisting.

Unsubscribe Handling

It is important to give your recipients the ability to unsubscribe from emails. First, it is required by the CAN-Spam Act. Second, if you don’t give them this option, they are more likely to click on the spam complaint button, which will cause more harm than allowing them to unsubscribe. Finally, many ESPs look for unsubscribe links and are more likely to filter your email if they don’t have them.

Mailgun customers have the ability to include an unsubscribe link automatically in your emails. We give you the ability to link the unsubscribe to a certain campaign, mailing list or make the request global to your domain. You can access this data through the Control Panel, API or via Webhooks. In addition, we will automatically stop sending to email addresses that have unsubscribed. It is possible to remove addresses from the flagged list in your Control Panel or through the API.

Recipient Engagement

In addition to processing bounces, complaints and unsubscribes, ESPs measure your reputation through the engagement of your recipients. If recipients are opening, forwarding and replying to your emails, it will improve your reputation. This is what makes ”do-not-reply” emails so offensive. At many ESPs, it is also helpful if recipients add your email address to their address books.

So it’s important that you write emails that your users engage with.  Obviously, this is good for business, but it also means that more of your email will get through ESP filters, which creates a virtuous cycle.

For Mailgun customers, you can track opens and link clicks with our Tracking and Campaign functionality (see our User Manual for more information). You are free to create as many campaigns as you want and use them simultaneously for A/B testing. In addition, Mailgun is built to receive and parse emails efficiently. So there is no excuse to not allow your recipients to reply to your emails. Email is not a billboard – it is a conversant technology.

Whitelists and Other Deliverability Tools

While not required, it is a good idea to sign up for whitelists where available. A whitelist (that’s the opposite of blacklist) is like having your email pre-cleared by an ESP. Not all ESPs have them and they usually require some history of sending before they will allow you to sign up. Also, most of them require that you be on a dedicated IP address. Word to the Wise provides a list of whitelists along with feedback loops.

In addition to the whitelists provided by the ESPs, Return Path has a certification program that whitelists you at many of the major ESPs. In addition, the certification enables images to be displayed by default at many ESPs (which is not usually the case). Return Path also provides seed lists (so you can test if your email is being spam filtered) and campaign preview (to see how your emails are being rendered across ESPs and test for spammy content). You can read more about the benefits on Return Path’s website.

For Mailgun customers with dedicated IP addresses, we can register you for whitelists. Also, you can set up a test inbox and use our spam filtering technology to provide a “spamicity” score for your outgoing emails, which lets you see their propensity for being filtered. Finally, all accounts with dedicated IP addresses are qualified for Return Path certification. Our technology has been pre-vetted so the only remaining step is for them to monitor your sending in order to certify you.

That’s it for this time!  In our final installment, we’ll discuss how to write emails that don’t get flagged as spam.

In Part 1, we gave an overview of how to maintain a good reputation with email service providers (ESPs) and how to host your email infrastructure if you want to manage email deliverability yourself. Part 2 discussed how IP addresses, sending volume, DNS and authentication affect deliverability. Today, we’ll look at important topics like how to maintain email lists, how to handle bounces and spam complaints.

Managing Mailing Lists

The No. 1 reason we see people get blocked by ESPs is because they have a bad mailing list. Don’t purchase your list or scrape websites for emails. It’s the easy way out and you will pay the consequences. Most of these lists have bad email addresses and include spam traps. ESPs are very good at recognizing bad mailing lists.

You should only send emails to people who have opted in to receiving your emails on YOUR website. In addition, you should send a verification email with a link that confirms their subscription (double opt-in) to make sure their email address is correct and that they are indeed the person that signed up. If everyone did this, the world would be a better place.

You should have your Privacy Policy easily accessible on your website. In addition, you should have a place on your website where users can unsubscribe from your mailings, in addition to a link in every email you send (we’ll cover unsubscribe handling in a future post).

If you’re a Mailgun customer, we give you information for bounces, unsubscribes, complaints, opens and clicks so you can modify your mailing lists appropriately. In addition, we automate a lot of the work by keeping track of recipients that have unsubscribed, bounced or complained and stopping future deliveries to those recipients. We give you various levels of unsubscribe granularity so your recipients can unsubscribe to all emails from the domain, just that mailing list or just emails with a specific “tag” (which you define).

Bounce and ESP Feedback Handling

A big part of maintaining your email reputation is processing bounces properly. While most major ESPs give bounce replies “on the wire” during the SMTP session, there are some that send bounce messages via email. In order receive these emailed bounce messages, you must have the appropriate return path header included with your email so recipients know where to reply with bounce information.

You must also process this bounce data and act accordingly. In addition, many ESPs will soft bounce your initial attempts at delivery. This is also called grey-listing or throttling. If you continue to send emails to bad addresses or you do not listen to ESPs feedback, you will get filtered and eventually your emails will be dropped.

For Mailgun customers, we automatically processes bounce information and react accordingly. A good portion of Mailgun’s technology is devoted to the parsing of this feedback and adjusting your sending in accordance with this feedback so you maintain a good reputation.

If we receive a hard bounce, we will stop sending to that address immediately and will not attempt future deliveries to that address. We will stop sending to an address after multiple soft bounces, according to the ESPs’ guidelines. It is possible to remove addresses from the flagged list in your control panel or through the API, in case it was a temporary issue.

If you are not a Mailgun customer, you should make sure that your provider takes similar actions, or that you have sent up these bounce handling procedures to ensure that you maintain a good sending reputation.

Feedback Loops and Spam Complaints

Most of the major ESPs (other than Gmail) provide feedback loops through which they give you information about spam complaints. Here is a thorough list from Word to the Wise. It is important that you sign up for these feedback loops and pay attention to the feedback you are getting. If you ignore this feedback, ESPs will throttle you and eventually block you completely.

Mailgun registers all of our IPs for these feedback loops. Customers can access this information through the Control Panel, the API or Webhooks. In addition, we process spam complaints automatically and will stop sending to email addresses after a recipient complains. It is possible to remove addresses from the flagged list in your Control Panel or through the API.

That’s it for this time!  In part 4, we’ll discuss unsubscribe handling, recipient engagement and whitelisting.

In Part 1 of this series, we gave an overview of how to maintain a good reputation with email service providers (ESPs) and how to host your email infrastructure if you want to manage email deliverability yourself. Today, we’ll look at the important topics like how IP addresses, sending volume, DNS and authentication affect deliverability.

IP Addresses and Sending Volume

If you send a lot of email (greater than 50,000 per week), it is a good idea to have a dedicated IP in order to isolate your reputation. If you share your IP, you share your reputation with those other senders. In addition, ESPs rate limit your emails based on the IP. So if you are a high volume sender you should consider getting a pool of IPs. However, your reputation can also be hurt if you are not sending enough volume consistently from an IP, so it’s a tricky balance.

If your email sending is volatile with large spikes of volume, ESPs may assume those large spikes are spam. Also, if your overall volume is too low, they won’t acknowledge your reputation. Generally, if you send less than 5,000 emails per day, a shared IP may be the right solution.

The other thing to consider is using separate IPs for your bulk and transactional mail if you are sending high volumes of email. There are a couple reasons for this:

• Delivery of time-sensitive transactional emails may get queued behind a large batch of bulk/marketing emails.
• Your transactional mail will be affected by the reputation created by your bulk/marketing mail.

Even if you have a clean IP address, you need to warm up the IP gradually. This means sending emails at a low rate initially and then gradually increasing that rate, taking into account ESP feedback. If you send a ton of emails right away, they will get filtered or dropped by the ESPs. In some cases, they won’t even tell you they are dropping them.

If you decide to use Mailgun, we offer both shared and dedicated IPs. We constantly monitor the traffic on these IPs. So even for shared IPs, you can be comfortable that your reputation is not unduly influenced by others. We also offer pools of IPs for high volume senders. In addition, we have queuing algorithms that gradually warm up your IPs. Our sending rates automatically increase over time as your IP warms up. Finally, we separate our sending queues for each domain you set up at Mailgun, which mitigates the need for multiple IPs for different types of traffic.

DNS

Your email reputation is not only tied to your IP, but your domain name (DNS stands for Domain Name System) as well. You should keep this in mind as you set up your email infrastructure. For the same reasons as above, it is a good idea to have separate domains or subdomains for your marketing, transactional and corporate mail. We suggest that you use your top level domain for your corporate mail and use different domains or subdomains for your marketing and transactional mail.

While it is not required to use the same domain in the From field of the message as the actual domain sending the message, it is highly recommended. Hotmail is especially finicky about this requirement and has a higher propensity to filter your messages to junk if the two domains do not match.

You should also make sure that you use a well-regarded DNS provider and that you publish all of your contact information in the WHOIS record. If you hide your contact information through a proxy, ESPs may take that as a signal that you are spamming.

Also, make sure you include the appropriate records at your DNS provider for authentication (see below). While it’s not required to point mx records to the same domain as you are sending from, it is recommended. There are email providers (albeit, a minority) that will check if mx records for the domain are valid before accepting email.

For Mailgun customers, you have the ability to create multiple domains or subdomains very easily. You are free to create multiple domains and subdomains for each of your transactional, marketing and corporate email. Each domain has an isolated queue, so your transactional emails won’t get held up by your bulk mailings.

Authentication

It is very important that you use the appropriate authentication methods with your email. If you do not authenticate your email properly, ESPs will assume you are spamming and will filter or just drop your email.

The common types of authentication are:

• SPF
• DKIM
• DomainKeys
• SenderID

Mailgun uses all of these types of authentication. When you sign up for Mailgun, we provide the appropriate records for you to include at your DNS registrar. We also provide a verification button you can use to make sure that your records are set up correctly.

That’s it for this time! Let us know if you have any questions in the comments. In part 3, we’ll discuss managing Email Lists, handling Email Bounces, and Feedback Loops.

Sending email is easy. Having it delivered consistently is hard. This blog series is a summary of email best practices that we have learned from managing mail servers for thousands of customers and from sending a lot of email.  The objective is to help outline what you need to do to have your emails delivered whether or not you use Mailgun. Of course, if after reading the guide you decide that you have better things to do than maintain email servers and manage email deliverability, we’d love to help!

In this guide, we will not focus on the content of email messages. It’s as simple as this: send something people want (to paraphrase Y Combinator’s motto). That’s probably the hardest part, so apologies for that huge caveat. Instead, we’ll focus on the infrastructure and monitoring of email so that if you are sending something people want, they will get it, and if you are not, you will know about it and hopefully change that.

In Part 1 of this guide, we look at how to maintain a good reputation with email service providers and ensure that your email infrastructure is hosted properly for maximum deliverability.

Reputation

One of the most important assets you have in the email world (much like the real world) is your reputation. If you do not have a good reputation tied to your domain and your IP address (IP used herein for abbreviation), your email will not reach your recipients’ inboxes. Due to its popularity and its unique ability to push information to users, email has been overrun with spammers (as if you didn’t notice). Depending on your definition, approximately 90 percent of all email is spam, according to MAAWG. Due to this, email service providers (ESPs) like Gmail, AOL, Yahoo and MSN/Hotmail have declared an all-out war on spammers. This has made our inboxes more pleasant places. This also makes it very important to manage your email reputation. If it is not impeccable, you will get caught in the ESPs’ spam filters.

A good analogy for your email reputation is your personal credit score. Obviously, a bad reputation will hurt you. However, not having a reputation will also hurt you. If ESPs don’t know you (or more specifically your IP and domain) they will assume the worst and filter you, at least initially. It’s tough to blame them given all the spam out there. Due to the importance of reputation, a significant portion of our discussion on best practices revolves around building and maintaining your email reputation.

Beyond making sure that the infrastructure is properly set up (more on this below), it’s important to listen what ESPs are telling you if they are not delivering your messages (Mailgun provides this information to our customers. Check with your email provider or review your logs if you are not a Mailgun customer)

• Are emails being delivered and if not, why?
• Is a recipient ESP throttling your traffic and why?
• Are messages bouncing due to incorrect domains or stale addresses?
• Are recipients unsubscribing or complaining of spam?
• Are recipients engaging with your emails by opening them and/or clicking on links?

You should use all of this data to make sure that you are complying with ESPs’ guidelines and adjust your email sending to stay in their good graces.  At Mailgun, we use queuing algorithms that adjust sending rates based on feedback from recipient ESPs. You should consider this as well.

Some email service providers use FUD (fear, uncertainty and doubt) about email deliverability to sell you a deliverability fairy that magically gets your emails to the inbox. This is most definitely not the case and your actions, as the email sender, play the biggest part in good deliverability. If you’re a Mailgun customer, we give you all the tools for establishing a good sending reputation, but it’s ultimately up to you to send emails appropriately.

Regardless of how you send your emails, if you follow a couple rules (along with properly authenticating your email), you will most likely build up a great email sending reputation:

• Only send emails to people who have signed-up to receive them from your website/application/service, and always first send a confirmation link to confirm their address is correct (aka “double opt-in”).
• Track your email and adjust your sending based on feedback from ESPs and recipients (e.g. don’t send additional emails to recipients that have unsubscribed or complained of spam).

Hosting

A brief note on hosting: since hosting technology is changing so quickly, it will likely be out of date in a few months. Due to the way that ESPs currently assess the reputation of email senders based on IP address; large, virtual cloud environments are generally not the best environments for email. This is not due to the reliability of the cloud itself, but the methods that ESPs use to fight spam. To overcome these limits:

• The IP address should be static so that your domain(s) & IP address(es) build a reputation together. Also, some more strict recipients ESPs may require whitelisting your IP address. Unfortunately, these should be IPv4.
• The IP address and surrounding IP addresses should have a good reputation. This is often a problem at large cloud environments due to their ease of use and lax monitoring (which is inviting to spammers).
• Mail Transfer Agents should ideally be on  dedicated (non-virtual) machines, optimized for I/O.

How do we host our email infrastructure to get the best results? We host Mailgun mostly on dedicated servers. We do use cloud servers for some of the infrastructure (where it makes sense), but for most of Mailgun we like large, robust, dedicated machines for the reasons outlined above.

We use dedicated IP addresses in large subnets and we do background checks and extensive testing on our IP addresses. Because they are in large continuous blocks, they are less likely to be affected by other, external IP addresses. ESPs and blacklists occasionally block entire subnets if any of the IPs have questionable reputations. So even if your IP is clean, it might be blocked because of surrounding IPs. Larger subnets mitigate this risk.

We dream of a day when IP reputation does not matter and we can rely on domain reputation, but unfortunately we are not there yet.

That’s it for today.  Stay tuned for more email deliverability tips.  Next time, we’ll discuss managing IP address, DNS and Authentication to maximize deliverability!