As conversion optimization (or sometimes called conversion rate optimization or CRO) has started becoming more commonplace in the online marketing industry, I’ve noticed that the terms “split test” and “AB test” seem to be interchangeable. However, the way I see it, this is not the case. Each of these terms has unique meanings. So, I thought I would take a moment to clarify each of these terms.
To start, here are some definitions from Wikipedia:
Wait a minute, these are exactly the same definitions. So, what’s the problem here?
OK, so here’s how I would change these definitions:
To help clarify this, here’s an example:
|Control Ad||Variation Ad|
To split test a landing page I would send 50% of the traffic to one landing page design (“Control Ad” above) and the remaining 50% of the traffic to different landing page design (“Variation Ad” above). Both of these landing pages would have the same conversion goal (fill out a form, click a button, or whatever). However, the landing pages could have different text, different headlines, different photos, different buttons and more importantly a unique URL.
Then I would take the winner of the split test and use it as the control sample in an A/B test.
For an A/B test I would test the winning control sample against one variable change made to the sample (for example, change the color of a button from orange to green).
|Control Ad||Control Ad with variable change (a green button)|
From there, we would move on to the more complicated multivariate test.
Here’s a definition from Wikipedia:
For example, using the control sample we could test multiple variables (such as button colors, headline text, number of form fields, photos, color schemes, and more) all at the same time. Now, testing several variables at the same time requires quite a bit of traffic to the page being tested. The more variables you want to test simultaneously the more traffic you will need.
So, I would like to recommend the following testing steps:
1) Split Test – Use a split test to find out if one design direction is better than another.
2) A/B Test – Use an A/B test to test one variation of a variable on a single page.
3) Multivariate Test – Use a multivariate test to test multiple variations of multiple variables on a single page.
The bottom line for any of these conversion optimization testing methods is to increase the conversion rates of your landing pages. Conversion optimization testing is not a one-time thing, it’s a process that requires continuous improvement.
Test everything and anything, and test often.