The Power of A/B Testing

Suppose you had a tool in your bag of marketing tricks that would allow you to continuously monitor and improve engagement with your audience. One of the simplest techniques to employ to this end is A/B testing, also sometimes called split testing. This technique is equally effective with everything from web site content to emails and social media.

HOW DOES A/B TESTING WORK?

A/B testing is simple in concept, but it does need to be applied thoughtfully. Here’s a simple example. Suppose you are sending a group email advertising a 30% off sale. You need to make up the regular version of that email, using your typical verbiage. See the example below.

Suppose you would like to test if changing the colors for the ad makes a difference in response rate, so you also produce another version with a different color combination announcing the same sale.

To run the split test, you would send something like 100 emails which have the black/yellow combination to one group of people and the one with the purple/yellow combination to a different group of 100 people. Once you have launched the trial you will need to monitor the clicks that you get for each ad. For purposes of this example suppose you release the ad in the morning and by the late afternoon there is enough information to capture a meaningful sample response. You now know there is a clear preference of which version gives the best response to the CTA (call to action, i.e. “click for details”). Now you can confidently release the preferred version to your entire mailing list and reap the reward of a higher response rate. This is a simple, manual example and you’ll want to invest in some software to automatically record the results and help with the analysis as you get more sophisticated.

IMPROVING YOUR ENGAGEMENT

Just imagine how knowing this type of information could enhance your marketing effort and response rates when applied to web pages, product designs, apps and social media. In the example, the only variable was font color, but this technique could be applied to which headlines, subjects, or designs are most favored by your audience on just about anything. In fact, it is one of the most used tools in marketing because it is easy and very flexible. Although this technique is often used to compare two variations, it can be expanded to three or more (multivariate) investigations. Perhaps the best thing about A/B testing is that it gives you fact-based information to help with decision-making and you can get this information quickly.

INCREASING ROI

Armed with A/B testing allows you to positively affect your ROI. See where on your web site your prospects spend the most time and find ways to increase their time on your site. This also helps with SEO. You could use A/B tests to see if testimonials on your shopping pages increase conversion rates. Does shortening the order process result in fewer abandoned shopping carts? Test your CTA’s to see what is most effective.

BEGINNER TIPS

Test one element at a time

By limiting your test element to a single item it is very easy to see the effect of making the change. However, if you are testing multiple variations in a single test, there is an increased chance of confusing which variables have caused specific effects.

Establish a goal

Once you have decided the variable you plan to test, establish clear goals for this test. What do you want to get out of this test. Be as specific as you can.

Create the variant

Keep this simple at first. A great place to start is to test CTAs. Make two identical pages with different CTAs and see which one performs better.

Decide on a schedule

Set an end point for the test. You want enough data to be meaningful. Data groups of 500 to 1,000 points usually contain enough data that the analysis is meaningful.

Analyze and act on results

After all the data is collected, sit down with your team and any analysis software and share the information. Get agreement on the results and whether or not you’ve met the test objectives. Note any unexpected data on the results, like shares and other engagement metrics. These may trigger other A/B test proposals. Implement the proposed change and then move on to the next element that you would like to test.

A/B testing is a simple and easy to use technique. It should definitely be part of your bag of tricks for improving the ROI of your marketing efforts. Perhaps the biggest mistake of A/B testing is that of not using it enough. Get creative, get familiar with the techniques, then apply them for any change to your on-line presence. You’ll be glad you did and your customers will be too.