Testing is one of the fundamental tools employed to improve the efficiency of marketing campaigns.
A/B or Split-Testing are two names for the way the results of one element of a campaign – an email, a landing page or a banner ad – is compared with another to determine the effect a specific variable has on the outcome.
The most common uses of testing is to evaluate headlines, offers, design elements, color, layouts and other treatments that are part of the user interface. Anything that can be precisely measured can be used to judge the efficiency or ability to yield a response.
With A/B Testing there should always be a control group against which the performance of one or more test groups are compared.
Start with a Test Subject
The first item of business will be deciding what needs to be tested. Then, a few variables will be created that allow data to be collected. Examples of subject and variables are:
Layout. Move the registration form to a new location, change where visuals are located.
Headlines. Test specific words in the headline. Alter the size of the type, even the color.
Copy. Create different versions in the content. Or try one with subheads and one without.
Try to limit the test to no more than four variables. And, don’t combine variable or the data you generate will be of little use when it comes to determining the single most important factor responsible for the difference in results.
Randomize For Meaningful Results
In other words, you must conduct your test on a level playing field.
That means you must run the test with all of the different versions in parallel and randomly assigning a treatment to each target. This helps compensate for time-sensitive variables and distributes the different variable evenly across all populations.
A Practical Example
A company which sells coffee mugs would like to improve sales. The current promotional email has yielded mediocre results. An offer is being toyed with in the minds of the marketing team. But they would like to find one that is not too costly to maintain the profit margin.
Two offers will be tested against the control. (An email without an offer of any kind.)
Sample A will be an offer of FREE shipping. Sample B will be an offer of a $1.00 coupon for the purchase of ground coffee at the store. The offer is reflected in the headline of both emails.
It is critical that the difference between the A and B emails be limited to the offer. All other elements must be the same. Same visual, same typestyle and size, same layout, and order mechanism.
The test is sent out to three groups of 1500 each and yields the following data:
From the data, Sample B emerges a winner.
Sample Size Matters
The sample size is the number of people who received that sample. The larger the sample size, the more useful the results are in terms of predicting the results of a full scale campaign to hundreds of thousands.
There are so many variable to test in any example of marketing elements. Here are some examples:
- Placement of a subhead telling the reader that registration is FREE.
- Button color – which works best, red buttons or green buttons, etc.
- The User Experience – two emails are sent out, one offers a link straight to the online store, and the other email has a link that takes the user to a landing page. Which produces higher sales?
- Deadlines: One email has an offer that expires in a week; the other email has no deadline.
- Free gift with purchase vs. buy-one-get-one
- Control vs. FREE upgraded shipping service (faster)
- Zero % financing for 6 months vs. $XX discount
The possibilities are endless. And you can learn so many things about your marketing that you can seriously increase efficiency and drive profits.