+44 (0)1296 713635 info@marketwise.co.uk
  1. Home
  2.  » 
  3. Email Marketing
  4.  » A/B Email Testing

How important is A/B email testing?

Very. Unless you personally know the majority of your subscribers and understand their hot buttons, A/B testing is a must for most email campaigns. Even if you are mailing a regular audience, it still pays to test different versions of your message everytime. Small improvements soon grow to become quite significant.

A / B email testing: stop guessing and start analysing

Before you start, make sure your A and B groups are random and representative. If one group includes way more Gmail addresses, or the B group many more Chinese emails for example, your results could well all be invalid.  At Marketwise, we aim for A and B groups to be around 1000 contacts each, taken randomly from the audience.

A/B testing for email marketing

Subject line A/B testing

Perhaps the most common and easiest type of A/B testing is for email subject lines. Just because its easy, doesn’t mean it is important. In fact it could be the most important A/B testing you do.  Improving the open rate of your emails is the single most important metric to focus on. Subject lines are critical to this. So what factors should you test?


Our recommendation, as stated on this page, is to aim for around 40 characters. However, this is a recommendation, not a hard fact, so subject line length is definitely something to play with. Just keep in mind that the first 40 characters are your most valuable, above that, truncation can occur.  Larger screens and higher resolution displays are allowing more space, so it can be worth experimenting.

    • A: New cell biology breakthrough
    • B: New cell biology breakthrough saves hours in the lab

Statement or Question?

People respond to statements and questions in different ways. They invoke different responses in the reader. A simple statement might invoke interest, a question can create curiosity.  Both are good responses to provoke in your reader and can encourage higher email open rates.  This makes them great candidates for A/B email testing.

    • A: New Firewall now available as a container
    • B: Heard about the containerized Firewall?


Often hailed as the ultimate way to spark additional interest, personalization should be handled carefully and always A/B tested. The main tag used to personalise is First Name. The reason for caution is that some people do have quite long names! These people might commonly abbreviate their name, but unless you know this, it can make personalised subject lines a little hit and miss, impacting the length badly.


    • A: Maria-Elisabetta, try our new methods and protocols database
    • B: Try our new methods and protocols database

Message Content

Now lets look at the message content and how to use A/B email testing to improve click through rates.

Call to Action buttons

Two main options to test are locations and also the style. Most email marketeers agree that having the CTA “above the fold”, ie seen without scrolling, is a good idea and we would agree. A second CTA button at the end could either confuse the reader (unless they are clearly identical), or make it more convenient to click.  So testing an additiona CTA further down the message can make a lot of sense.

The style of the button can make a big impact. Always be consistent within the same email, but make sure it stands out from the background.

Rich messages

With so much rich content available, including images, sounds and videos, its tempting to throw the kitchen sink at your reader, in the hope something sticks.  Possible responses to this approach could be:

    • Wow, so much information, this is great!
    • Oh, too much information, I’m skipping this.

So always A/B test before you use the latest widgets and newly available options. Our advice is to keep it simple.

Plain messages

An often under used approach (in our opinion), is the simple text based HTML message.  Do you really need that banner image that is just for decoration? Maybe it would work better without it?  Smaller size emails without images are often less likely to trigger spam filters, giving another edge.

Marketwise A/B testing

We hope you liked our guide to A/B email testing. Marketwise includes Subject Line A/B testing for free in all our offerings and we strongly encourage you to take advantage of this option.