A/B Testing Q & A

Today, our Project Manager, Shanon Solava, answers questions about A/B testing.

Shanon Solava, Glowhorn Project Manager

Shanon Solava, Glowhorn Project Manager

What is A/B Testing?

A/B Testing is comparing two versions of a website to each other to see which performs better. A specific section or component of the website is chosen to analyze. In other words, A/B testing is a data collection tool to measure and validate design changes and marketing strategies. A/B testing is typically implemented to improve website conversion rates and improve the user experience.

Why should I use or implement A/B testing on my website?

A/B testing helps you make data driven decisions regarding your website. If you are asking the question, can my website be improved? Then A/B testing is for you.

On one hand, you can THINK your website is converting because your last campaign was successful. Sure, you met your campaign goals and your sales goals. Business is good.

On the other hand, you can KNOW what components on your website are working the best. You can know the level and pace of sales are on target based on data. You can pick an aspect of your website to focus on and make changes that you think will improve this area. You can test your hypothesis.

How does A/B testing work?

When an A/B test is implemented, half of your website visitors are shown variation A and half of the visitors are shown variation B. For example, if variation A is the control group and you are testing ‘buy now’ color button options, the control group would see buttons that are blue. Variation A equals blue buttons. Variation B is a button in orange. Half of your visitors would see orange buy now buttons and half would see blue buttons.

The A/B test can track how many people click on each button and if the click resulted in a sale. The results of a similar A/B test are typically implemented on an ecommerce site to analyze what color of button results in an increase in sales.

What can I test?

Example website components that can be tested include:

  • Page titles
  • Blog post headlines
  • Images
  • Forms
  • Form fields
  • Calls to action
  • Button colors
  • Page layouts
  • And more

What is the A/B Testing Process?

First, you start collecting data or review existing data. If you already have Google Analytics set-up, you will be able to rank the most regularly visited pages of your site.

Second, you can develop a scorecard containing your goals and metrics.

Example goals are:

  • Increase sale leads
  • Increase ecommerce conversion rate
  • Attract a specific type of buyer
  • Increase the amount of time spent on the site
  • Increase social media sharing
  • Balance seasonality of sales/visits throughout the year

Example metrics include:

  • Total amount of visitors
  • Total amount of visitors for a specific page
  • Number of button clicks for a call to action or purchase button
  • Number of email sign-ups
  • Number of downloads (for example, of an ebook)

Next, it’s time to set up tests relating to your goals and metrics. This is where your A/B testing plan starts to come together.

Example tests include:

  • Free offer lead page designs
  • Messaging for specific areas of a page
  • Calls to action
  • Email sign-up options (location, message, etc)
  • Content, in general
  • Navigation
  • Checkout process/steps
  • Social sharing buttons

Your A/B testing goals, metrics, and tests should directly relate to your marketing plan. For example, your content should speak directly to your key personas. Your design variations can be a custom creation or you may be able to use a visual editor to make simple color changes. Either way, you need to use your A/B test variations for a specific amount of time then analyze the results.

In summary, the A/B Testing Process steps are:

  1. Start collecting data/review existing data
  2. Develop a Scorecard
  3. Develop specific test strategies
  4. Design your variations
  5. Launch your test
  6. Analyze your results