Split Test vs. AB Test: Is there a difference?

As conversion optimization (or sometimes called conversion rate optimization or CRO) has started becoming more commonplace in the online marketing industry, I’ve noticed that the terms “split test” and “AB test” seem to be interchangeable. However, the way I see it, this is not the case. Each of these terms has unique meanings. So, I thought I would take a moment to clarify each of these terms.

To start, here are some definitions from Wikipedia:

• Split Test – A split test is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates.
• AB Test (or A/B Test) – An A/B test is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates.

Wait a minute, these are exactly the same definitions. So, what’s the problem here?

OK, so here’s how I would change these definitions:

• Split Test – A split test is a method of marketing testing by which a baseline control sample is compared to a completely different variation of the same sample in order to improve response rates.
• AB Test (or A/B Test) – An A/B test is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable tests performed on the same sample in order to improve response rates.

Split Test Example

To help clarify this, here’s an example:

Control Ad	Variation Ad

To split test a landing page I would send 50% of the traffic to one landing page design (“Control Ad” above) and the remaining 50% of the traffic to different landing page design (“Variation Ad” above). Both of these landing pages would have the same conversion goal (fill out a form, click a button, or whatever). However, the landing pages could have different text, different headlines, different photos, different buttons and more importantly a unique URL.

Then I would take the winner of the split test and use it as the control sample in an A/B test.

A/B Test Example

For an A/B test I would test the winning control sample against one variable change made to the sample (for example, change the color of a button from orange to green).

Control Ad	Control Ad with variable change (a green button)

From there, we would move on to the more complicated multivariate test.

Multivariate Test

Here’s a definition from Wikipedia:

• Multivariate Test – A multivariate test is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time.

For example, using the control sample we could test multiple variables (such as button colors, headline text, number of form fields, photos, color schemes, and more) all at the same time. Now, testing several variables at the same time requires quite a bit of traffic to the page being tested. The more variables you want to test simultaneously the more traffic you will need.

So, I would like to recommend the following testing steps:

1)      Split Test – Use a split test to find out if one design direction is better than another.

2)      A/B Test – Use an A/B test to test one variation of a variable on a single page.

3)      Multivariate Test – Use a multivariate test to test multiple variations of multiple variables on a single page.

The bottom line for any of these conversion optimization testing methods is to increase the conversion rates of your landing pages. Conversion optimization testing is not a one-time thing, it’s a process that requires continuous improvement.

Test everything and anything, and test often.

For more information on how we increase conversion rates call us at 678-951-9506, email us at info at surgelabs.com or complete our FastForm.