The Importance of A/B Testing
The Importance of A/B Testing

The Importance of A/B Testing

Published June 6, 2016 in Marketing
Louie on a canoe
Summer of Louie
May 27, 2016
Why Magento 2
Why Magento 2?
June 15, 2016

A/B testing (also referred to as split testing) is the process of testing and comparing two different versions of a web page, to determine which “performs” better. Testing uses real-time data and statistics to help determine the effectiveness of proposed changes, resulting in a site that is fully optimized.


The Testing Process

  1. Ask something.
    Determine the question you want to ask.
    Ie: “Why is the bounce rate on X page so high?
  2. Collect relevant data. 
    Utilize analytics tools to determine what pages and content assets are doing well.
    Understanding your customer behavior will help determine which elements to test.
  3. Identify and document your goals.
    Determine which conversion goals will be used as metrics to measure the test’s success.
    Goals can be simply clicking on an add, purchasing a product, completing a contact form, etc.
  4. Formulate a hypothesis: 
    ie: “Updating the graphics on this page will decrease the bounce rate”.
  5. Create test variations. 
    There are multiple ways to create unique variations of each page. You can utilize A/B testing software such as Optimizely or VWO (Visual Website Optimizer).
  6. Put it to the Test: 
    Launch your A/B test in which the variation (version B) has implemented the proposed changes.
    Test this page against the original, and measure each page’s bounce rate.
    Visitors must randomly be directed to either page.
    Analytics tools will track their behavior to determine how well each page performs.
  7. Analyze results and draw conclusions: 
    If the variant page does better than the original, your test is a winner! If not, keep testing other variables until you reach your goals. Each test will reveal something different!

What to Test

The variables you decide to test will depend solely on your goals for the experiment. Once you determine what your end-goal is, the elements to test typically reveal themselves. Feel free to get creative and test subtle details. For a general rule of thumb, the table below lists out key variables to test.

The Importance of A/B Testing

A/B testing allows you to optimize your site on every level, making use of your existing traffic. It gives you an opportunity to make careful, calculated changes while measuring the impact on conversions. You eliminate the risk of guesswork and can prevent immeasurable damage to your site.

Testing one change at a time will help you pinpoint the root – or roadblock – to increasing conversions. You can continue consistently improving and optimizing your site, without the harsh repercussions of launching without being tested.

Things to Keep in Mind

As we’ve been testing, a few key lessons have stuck out to our team.

Always test the control and the variable simultaneously. Split your traffic between the two versions; otherwise, your results may be skewed.

Don’t read too much into the initial results. If you conclude too early, you may find that the initial results weren’t sustainable. You can find online calculators to determine your test’s “statistical confidence.” (Which tells you how credible and relevant your results actually are).

Allow the results to shape your strategy. Too often, we go into a test expecting a specific outcome; when it doesn’t happen that way, it’s easy to disagree with the results. Just remember that the overall goal is a better conversion rate, regardless of what your personal preference may be. Every test can have 3 results: no result, a negative result or a positive result.

Why Classy Llama A/B Tests

Classy Llama tests, because every good marketer does. We love to do good by our clients. One way we can do this is by analyzing our customers’ sites and running A/B test ideas to improve it.

Testing out ideas is important because what may have worked for one site may not work well for another. There is no one right answer when it comes to the perfect mix for a good UX and good conversion rates. The only true way to know what works with your customers is to run a set of A/B tests and measure them against your current conversion rates.

A/B tests help you get to know your customers better and drives in providing them with the best UX. This, in turn, improves conversion rate performance and makes for a very happy client.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related postsView all
December 18, 2017

Meet Customers Where They are in the Buying Process – The Decision Path

Why Targeted Messaging Matters You are looking for new ways to get more customers. If you could only find people at just the right moment with […]
October 9, 2017

Canonical Tags: Did You Get It Right on Your Magento Site?

A few weeks ago, a llama posed a seemingly simple question: “Does anyone know the details of canonical URLs and pagination?”. But here’s the thing about […]
August 18, 2017

The Ultimate Guide to On-Page SEO for Product Pages

The Ultimate Guide to On-Page SEO for Product Pages Whether you’ve been selling online for years or are just getting started in eCommerce, it’s important to […]
Most Recent PostsView all
February 12, 2018

Display Configurable Product Price Ranges in Magento 2

Today we’re going to look at the advantages of using configurable products and how we can update how their prices display on Magento 2 product listing […]
January 11, 2018

Selling Bundles the Right Way on Magento 2

Magento has a variety of product types to help you find the right way to sell your products. There are downloadable products for digital items (such […]
December 20, 2017

What does ROAS Really Mean?

ROAS (Return on Ad Spend) is a popular metric that can help businesses evaluate if their campaigns are successful. It’s a simple calculation of ad spend […]