How Do I Perform An Effective A/B Test?

If you want to give your conversion rate a boost, an effective A/B test is the best way to do it.

An A/B test lets you see how your audience responds to different variables, and gives you all the data you need to create an experience that generates the most conversions. You can’t really argue with that, can you?

Here we’ll have a closer look at what an A/B test is, and give you some tips on how to use one to improve your conversion rate.

What is an A/B test?

An A/B test compares two versions of the same thing to determine which performs better. You may have heard it called ‘split testing’ or ‘bucket testing’ – don’t worry, they mean the same thing!

In a nutshell, A/B testing is an experiment based on your users' behaviour. Users will be directed to one variation (either the control or the challenger) and their actions will be analysed to work out which is more effective.

 A/B Testing Example

*Courtesy of conversionxl

What can you A/B test?

Pretty much anything on your website that affects visitor behaviour can be A/B tested. Here are just a few things you could try:

  • Headlines
  • Sub-headings
  • Body text
  • Testimonials
  • Call to actions (text/buttons)
  • Links
  • Images
  • Page formatting
  • Social proof
  • Media mentions
  • Awards and badges

You could also try more complex changes; adjust price structures, offer different promotions/free trial lengths, free/paid delivery, general navigation or the overall user experience. These can be a bit trickier to complete, but can make a massive difference.

Why is A/B testing important?

At FABRIC8, we believe that testing is vital. If you don’t test, how will you know if any changes you make are effective?

A/B testing takes the guesswork out of website optimisation and ensures that your new version will generate the most conversions and (ultimately) the most revenue.

It allows marketers to make informed decisions, and helps them learn about user behaviour. This is great, as it can fuel the rest of your content strategy and generate some great ideas for the future.

How do I perform an A/B test?

An A/B test can seem scary or intimidating, but it’s actually pretty simple. Especially if you’ve got the right software and a little CRO know-how.

To help you out, here’s a step-by-step guide to A/B testing that will help ensure you get the results you need to make your decisions.

1. Collect data

Before you start testing, you need to have a thorough look at your site to determine which areas you’re going to try to optimise. Otherwise you’ll be going in blind, which won’t be very effective!

Start by looking at high-traffic areas of your site. This will help you collect data faster and will provide a great basis for testing. Look for pages with low conversion rates, or high drop-off rates, as these are obvious candidates for optimisation.

The best way to collect data for an A/B test is by using analytics software. Google Analytics is a great place to start, while dedicated platforms like Mixpanel, Kissmetrics and Amplitude can give you deep insights into visitor behaviour.

2. Identify your goals

Conversion goals are the metrics you use to determine if the variation is more successful than the original. Without them, you won’t know if your A/B test has worked. As examples of conversion goals, you could try to increase:

  • Button clicks
  • Link click-throughs
  • Purchasing products
  • Sign up to email newsletters

Remember, don’t overreach with these, be realistic. You’re probably not going to boost conversions by 150% within a week, so don’t make that your goal.

3. Think up a hypothesis

At this stage, ask yourself why your page isn’t reaching your goal. Are there any elements you could change that might help with this?

We wouldn’t recommend changing the whole page. Instead, we’d suggest changing elements one by one to test their effectiveness. It could be a small change like switching a word, or a bigger one such as removing an entire section of your page. If you think your users will respond to it, then you should give it a try.

That’s your hypothesis. Again, keep it realistic – don’t hypothesise changing things you can’t or don’t have time to change – make sure you can facilitate these changes through your A/B testing platform.

Once you have your list of ideas, prioritise them according to the impact you think they’ll have and how difficult you think they’ll be to implement.

Then create timescales for each one. Some tests will only need a few weeks, others will need a month or more. It really depends on what you’re testing and the amount of traffic to the page. Be careful with this, as Google may penalise you if you test one element for too long!

4. Create variations

This is the fun part – use your A/B testing software to make the first of your changes.

Before it goes live, test that the change works from a visitor’s perspective. Remember, it should only be one element that’s changed. If adding a new image has sent something else out of whack and your visitors see it, it may well skew your results.

5. Start the experiment

Now just press ‘go’ and wait for your visitors to participate.

Each interaction with your control and challenger pages will be measured, counted and compared to determine how each performs.

You can pop in and check progress if you want, but often it’s best to wait until the end of the testing period to see which is most effective.

6. Analyse results

So, you have your data – now take a look at the results.

Your A/B testing software will present your data in a format that’s easy-to-digest, and let you know if anything is statistically significant.

Don’t worry if the test shows that your change hasn’t made a difference, or has even decreased conversions. This is a learning process to pin down what works and what doesn’t. Even if you see a negative result, it can be helpful when it comes to planning more A/B tests in the future.

7. Apply your findings

Imagine that your test has been successful and this encourages you to change all your buttons from red to blue. As soon as you know that’s the result, apply that change to your page.

Then, see if there’s anywhere else you can change this on your site. After all, if changing the colour of a button helps one page, it might help others too! Keep an eye on conversion rates for these pages over the coming weeks to see if the change has a positive effect.

Once you’ve done that, go to your hypothesis list and start your next test! You could even move on to other forms of testing:

  • Multivariate – testing several elements at a time
  • Experimental design – developing your own research method for an in-depth analysis of a specific element

A/B testing is an effective tool, but it's also a commitment that can require some expert knowledge to get right. At FABRIC8, we use A/B testing every day, so if you ever need support just ask!

However, if you want to go it alone, this step-by-step guide should really help. Take a look at your website and discover which elements you can test today!