When someone starts talking about testing, one of any number of things can come to mind. There is testing that ensures the site looks the way that it is supposed to. Some testing is done to make sure a website functions as expected. You can even have tested to confirm that a piece of code produces the correct result every time. Recently, we have become very interested in a completely different kind of testing called A/B and multivariate testing.

A/B and multivariate testing are two flavors of the same type of testing where the intention is to find a user experience that most effectively meets some merchant goal. The process we follow is to create two or more versions of the same page within your site and present them to the customer to see which achieves your goals more effectively. It is one easy and cost-effective way to improve a merchant’s site usability and conversion.

“But where do I begin? What should I test?”

As a merchant, you may say, “This sounds great, but my site has thousands of pages. Which one do I optimize?” To answer this question, a merchant must ask one important question: “What is my most important business goal?” In eCommerce, the most common goal is to increase customer conversion rates by some amount or to increase average order value by some amount to ultimately drive more revenue. Some other common goals may be to increase customer retention, to increase customer satisfaction, or to fix some “problem” with your website.

Once you’ve identified your goal, the next step is to analyze any metrics you can get around that goal. For the purpose of demonstration, we have chosen to discuss the topic of conversion levels. We utilize Google Analytics (GA) or any other conversion tracking tools you may have set up to set up GA goals, track them, and measure their current conversion rates. We also analyze where users are abandoning the site and use this information to identify the most promising page or pages to implement our testing.

Before implementing an A/B test, the next important step is to come up with a hypothesis about why you think users are leaving your site on this page.

Some steps you can take to brainstorm options are

• Research some user interface/user experience best practices.
• Research colors, fonts, and wording that may enhance your site’s usability and experience.
• Go to the problem page(s) and count the number of clicks it takes you to accomplish your goal. How many distinct pages have to load between when you enter the site and when you have completed your order?
• Find someone in your company who does not regularly use the website and ask them to be overly literal, then try to explain to them step-by-step how to use the “problem page.” Do this again on a comparable page on a more prominent shopping website like Amazon or eBay to see how your user experience compares holds up.

After you follow these steps, document your hypothesis, what you plan to test and why/how you expect it to impact your conversion goal.

Once you have come up with a hypothesis, it’s time to implement the test. There are a few great tools to help you get started with A/B testing. Our tool of choice is Optimizely, which is a great online tool that allows us to change the layout of your page, colors, text, fonts; launch the test, then track the results. We are able to do all of this with only a very slight modification to the actual site’s code which means the time to implement the tool is very minimal compared to some other methods which can be expensive or time consuming to implement. It also has built in analytics and works with Google Analytics to track conversion and other more comprehensive metrics.

We will work with you to implement the test and allow you to see the different versions that will be displayed to different groups of customers. Once your test is running, it is important to let it run long enough to reach a statistically significant result. A good rule of thumb is to let the test run for at least two weeks and have at least 100 conversions. Optimizely will also let you know once a test has reached a statically significant result and will declare a winner.

If the test version has won, it’s time to implement those changes and document the results. It’s important to keep documentation of each test you’ve completed so that you can reference and utilize the data for future tests.

At this point, your test is complete but now that Optimizely is installed and you understand how to find and create candidates for testing, the amount of time to implement another test has been reduced. You can now begin to implement a more long term strategy for improving your conversion, working toward other goals, and continually making small but meaningful improvements to your site. Optimization is an ongoing practice that should be done repeatedly to keep your site as relevant, profitable and effective as possible. It is important to remember that not every test will be better than the original, but that is why testing is so important instead of making changes without information about how your specific group of customers will react.

How We Work with You

The Classy Llama User Experience team consists of expert designers and front-end developers who look forward to helping you improve your site’s user experience and increase conversion rates. We start with an initial review of your current site analytics and do some brainstorming around recommended areas of improvement. Goals are defined and a roadmap for testing is established before testing begins. Upon conclusion of each test, a detailed report including findings and recommendations will be delivered. Based on these test results, we will make adjustments and evaluate opportunities for future improvements.

For more information, email [email protected] or call (417) 866-8887.