How To Conduct A/B Testing?

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

As marketers we all understand the struggle of putting together landing pages, newsletters, cold email campaigns, and more. Mastering the content that will be most engaging to your audience is difficult as people respond to different things. With this in mind decisions should not be made based on a marketers assumption, that is why A/B testing is so crucial. But what exactly is A/B testing?

What is A/B testing? 

A/B testing is the process of identifying a section of content that can be changed. This can be on your landing page, newsletters, cold emails and more. The idea is that you present two versions of the content to the same homogeneous segment to see which version performs best. It is important to note that the measures of conversion will vary according to the nature of what you are trying to test. 

Each stage of your funnel is an opportunity to capture more leads for your business and so it is important to identify where your conversion opportunity lies to increase more traffic and convert more users. Marketers might choose to A/B test websites to better user experience or test different email copies and subject lines to explore where the conversion opportunities lay. 

User Experience 

User experience will look at any changes that can be made on the website to maximise conversions. This could be a simple change such as changing your website hero image. For user experience you will focus on creating two separate landing pages, version A, which will be your original version and version B, which will be the changed version. Both landing pages will be sent to the same segment to see which one gets the better results. 

Email Copy Or Subject Line

An email subject line is the most important part of email marketing. The subject line needs to draw the attention of the reader to make them want to open your emails and read more. A/B testing can be crucial to identify what subject lines are most engaging. Similarly, an email copy needs to outline the exact action you would like your reader to take. A/B testing will help you test different email copies to optimise your conversions. 

Why Should You Perform an A/B Test? Analysing Common Goals 

A/B testing helps marketers reach their goals. A simple change to your email copy or the layout of a webpage can help you to achieve your desired results! Let’s take a look at how A/B testing can help achieve your goals: 

Bounce Rate

Bounce rate is the percentage of users who visit your website and leave without taking any additional actions. A high bounce rate can indicate a number of things and so there is no proven way to fix it. Due to the complexity of different product offerings, what might work for one website will not necessarily work for another. A/B testing allows you to test different versions of the same website to find the version with the lowest bounce rate.

Website Traffic 

A simple change to the title of your blog or a different version of your website copy can increase the number of users who visit your website. Pick a variable you would like to test to represent traffic and see which version offers the best results.

Conversion Rate

Changes to the location of your call-to-action (CTA), the colors of the buttons, different fonts for your copy and even the text in your CTA can increase the number of conversions .

Better ROI 

Acquiring quality leads can be costly and time consuming. A/B testing can help optimize your website’s conversion on your existing traffic. Minor changes can result in a higher conversion rate and increased ROI.

How To Perform A/B Testing

Yes, there is a method to this madness! A/B testing should follow a clear structure to ensure the best measurable results. 

Research

The first step in A/B testing it to research and analyse the current data you have about the content you would like to change. You should be asking yourself, if the data that you am currently getting is inline with your goals? If the answer is no, research measurable changes that can be made. For websites, use Google Analytics to look at your users behaviour; Who are they? What pages are they clicking most on? How many interactions are you getting per page? Where are your conversion locations? Number of opens on emails sent by me?? How many replies did I receive? How many unsubscribes were recorded?  

This will help you establish areas that need to be changed to maximise engagement. Once you have established what needs to be changed, pick a variable to test. 

Pick A Variable To Test 

There are many variables that can be tested in conjunction with one another, to analyse the performance of your websites or emails. However the key is to focus on one variable at a time. Focusing on more than one variable can cause confusion on what exactly is giving you good results. Isolate your “independent variable” and start thinking about what you are going to change. It can be the design, wording, subject line, senders, personalisation structures, etc. Small changes make for more effective results. 

Create A Hypothesis 

Create a hypothesis backed up by your initial research. Analyse the data you have already collected and predict what you expect to happen once the changes have been applied. Your hypothesis needs to be tested against quantifiable results you have identified to represent your overall goals. Pick one measure that will equate to your dependent variable and state clearly what you expect this measure to be at the end of your test, make sure to include this in your written hypothesis. 

Create Your Versions

Now that you have identified both your independent and your dependent variables, create your controlled version and your challenger. Your controlled version will be the original piece of content that will go unchanged, and the challenger will be your variation.

Sample Group 

To make sure your testing is as controlled as possible, the segments need to be homogeneous. If you have a segment size of 500, the test should be split in half serving 250 people in that segment version A, and the other half in version B.

 If you are running A/B testing on your email list, it is recommended to select a small list of contacts to test with so you do not burn your resources. If you are focusing on testing elements of your website, you will not be able to control who is seeing your content, and so in this case it is recommended that you focus on testing both versions for the same amount of time. Depending on the tool you choose to use, all this can be set up for you automatically. 

Analysing Results 

Before identifying the winning version, identify how significant you would like your results to be in order to select one version over the other. Base your decisions on things such as confidence levels. The higher the confidence level the more you can be sure about your results. Once you have found the winning variation, serve the content to the rest of the segments. Analysing your results is an ongoing process as you should be continually gathering data to test both the direct and indirect impacts of your changes. 

A/B testing is a process that takes time to master. Identifying your goals and your resources will help you understand the key aspects that need to be optimised in order to increase your conversions. 

The key is to remember to run one test at a time so you are able to get measurable results. Now that you know all you need to know about A/B testing, why not try it yourself? 

Strengthen Your Weaknesses 

Why stop here? Check out or blog to learn more about SEO, social media marketing, industry trends and much more.