Email Marketing: Split Testing Improves Results “Up to 191 Percent”

Split testing of email marketing campaigns can improve open rates, click through rates and even purchase conversion rates. More email service providers are offering split test functionality, making it possible to upload two or more versions of a single broadcast and send them to random segments of your database.

A few days later, you compare the results. Ideally, sales or conversion tracking is set up, too, so that you are comparing the results that matter the most to your bottom line. Otherwise you will have to settle for comparing open rates and click through rates. Declare the better-performing version of your test as the “winner” and use it next time, perhaps against a new experiment, and so on, in a process of continuous improvement.

Test Various Elements of an Email Message

Almost any content element can be tested. These include:

• Subject lines
• Headlines and all other copy, including long copy versus short copy
• Product images or hero shots
• Text versus buttons for calls to action
• Number of products promoted
• Landing pages

Sometimes you will want to test just one thing. But you can also test a redesigned email template or landing page. This would be helpful in determining, for example, whether an email or site redesign is on the right track.

Test the Timing, Too

You’re not limited to testing just content variations. Sending the same content on different days of the week or at different times of day is another way to conduct split testing, for the purpose of determining when to send a new email newsletter. You could even send the same content to different segments of your database based on demographic or behavioral attributes (though this is moving from split testing into the realm of email segmentation tactics).

Remember, the data derived from split testing in not infallible. For example, a tested subject line might have failed because the control is a better match to the content of the email body, not because it was a bad idea. Design your experiment accordingly and think through the results carefully.

If getting experimental with half of your list feels too risky, some vendors let you set up your split as 75/25 percent or even 90/10 percent provided that your list is big enough (say 12,000 records). Just be sure to compare conversion rates, not raw conversions.

What you learn through split testing can be applied the next time you send the same message to your list or a different list. You can also incorporate the successful copy and design elements into your email template for future campaigns.

But sometimes doing better “next time” isn’t good enough. After all, you can only announce a new product or present a new offer to your list once. Most of us want to get it right the first time.

Test a Small Sample

Fortunately, some higher-end email vendors offer the ability to run your test with, say, 20 percent of your list (equally split between two separate messages) and then send the winner to the remainder. That’s 80 percent plus 10 percent from the test. In other words, you can send the higher performing email to 90 percent of your list and not just, say, 50 percent. Be sure that both versions of your test are transmitted nearly simultaneously, because time of day also affects response.

If your email service provider does not offer split testing, I recommend switching to one that does. Having said that, you can do this process manually. What is required is downloading your list, working some spreadsheet gymnastics, and then uploading multiple lists that you will use just once. Unfortunately, the same lower-end vendors that don’t offer split testing also tend to restrict the importing of new lists.

Split testing does mean a little more work, and it complicates deployment logistics. But not much. Usually, it is easier to think of two subject lines than to decide on one. Regular split testing will highlight decisions based more on reality than guesses.

Is Split Testing Worthwhile?

I have seen, firsthand, data for an email marketing campaign in which the click-through rate of one split test version outperformed the other by 191 percent. And the difference was just the headline used in the body of the email message. (Not surprisingly, this company uses this headline in all of its marketing campaigns now.)

Many email service providers say that split testing functionality is very underutilized by their clients. That means that you can get a jump on your competitors by implementing this tactic now.