1. SEJ
  2.  ⋅ 
  3. Marketing Analytics

A/B Testing: Why & How to Do Split Testing on Your Website

A/B testing (split testing) is the process of setting up two versions of one page, and testing which page outperforms the other. Here's how to do it right.

Did you know that you can test certain elements on your website on their own and determine whether they are effective for your own audience?

What if your call-to-action elements are not performing as well as you think they should?

This could be because of many reasons including changes in user behavior, audience shifts, and the overall way in how users interact with your website.

Making small tweaks and testing them is a critical process.

This testing process is called CRO, or conversion rate optimization.

It involves setting up scientifically sound tests on user interaction elements on your site, hoping to achieve a significant increase in performance when all of your tests are complete.

Tweaking changes on your website is more of a scientific process than a creative one.

Sure, you can randomly change something and hope for the best.

But, this seldom works out well and may lead to less accurate results because of the small audience sampling size.

Most of us have at least some idea of what we want to accomplish on our own sites.

But, what if you have a larger site in another industry that you’re not exactly familiar with?

How do you draw conclusions about what to change, and what not to change, if you don’t know how those users interact with the website?

Enter A/B testing.

What Is A/B Testing

A/B testing, also known as split testing, is the process of setting up two versions of a page, and testing which page outperforms the other with a slight change.

If you want to broaden your scope, you can set up more than one version of a page and perform what’s called a multivariate test. But, this is beyond the scope of this article.

For now, let’s return to A/B testing.

So how do you know what you should test and not test when you set up this kind of test?

What, exactly, should you test on your own site?

We will explore some advanced ideas where you can do just that later.

Your Site’s Goals Matter

This is a consideration when you are trying to work through how to get actual, physical conversions – people buying your service or product.

Conversions don’t just happen.

They require a significant time investment in studying how users perform on your site to make changes designed to elicit such action.

The brutal reality of digital marketing is this: you must know what the data is saying regarding changes before you can make the change.

While a savvy marketer will know what changes to make based on their experience, and thus cut the time needed for testing and analysis, which will enhance the performance of a website, a less-than-savvy marketer will do random testing that doesn’t always obtain the desired result.

Let’s take a look at an example:

One way you can make money on the internet today is through affiliate marketing. And how you do this is to offer customers a free product or service that you provide as an affiliate.

If they sign up for your list, you earn a commission for each sale that they make.

With enough information about your customer, you can build a relationship with them, because your customers will trust you.

All the while you are studying a good deal regarding your consumer, you’ll also learn a lot about the manner in which they use the website.

You’re providing them with advice that will help them achieve something they’re trying to do to solve their current dilemma: buying your service or product.

The elements that can have the greatest potential for return on your ROI during testing include:

  • All of your site’s CTAs (calls to action).
  • Your site’s overall background color.
  • The colors of your overall page elements.
  • The photography you use on-site.
  • Content and its structure on your site.
  • Any on-page elements requiring user interaction.

The elements on your site are critical adjustments that can result in significant increases in your performance.

If you think a conversion button is underperforming, you may test the color among a sampling of your site’s traffic.

You can’t just test a random element and hope it provides the results you are seeking.

You must approach A/B testing with the mindset that you don’t really know anything, and this test will reveal what you don’t know.

For example, you could run a test on how your site displays your phone number, using distinct elements among the same segment of your audience data sample, to determine what gets the best results.

You could run a content test, just to make sure that you’re not wasting your time and effort on content that just won’t work.

In this way, you can receive a real, empirical demonstration of just how well your material works in your own site.

Some sites can experience results exceeding 1,000% in final conversions just from one small change because the other variation was a poor implementation.

The Basics of A/B Testing

A/B testing is a means to conclude which page is most appealing to users, and which implementation results in the highest conversions.

Creating a successful A/B test involves research on several fronts:

The Size & Current State of Your Website

Most sites do not require A/B testing, especially smaller sites with smaller audiences.

You already know what’s the accepted best practice in your industry, and A/B testing will reveal tweaks you can make along these lines.

Large Site vs. Smaller Sites

If you have a larger site getting over 10,000 visitors/month, and has hundreds of pages, doing A/B testing can help you find out what will convert best among your user testing samples.

If you have an even larger site, A/B testing is a must.

Establishing a General Baseline

If your site has existed for any significant length of time, you want to establish a general baseline of your site’s traffic.

Once you have this baseline, you are ready to move forward with assessing website elements and what you need to change for an effective test.

Deciding Which Elements to Test

Really, this is a step that requires trial and error over a few tests in order to get right.

If you think red is a better color than blue for your industry, changing up your button color during the A/B test will reveal which one converts best.

If your phone call CTAs are green rather than red, and you don’t think it is communicating the proper action through color, the test will show what your users really respond to, rather than your assumptions about what they respond to.

This is an important distinction to make because unless you have talked to your customers or otherwise performed deeper analysis, you don’t know what they are responding to. And, everything you are doing is just a guess.

A/B testing changes this guess to a high precision instrument: after a successful test, you will know what your users are responding to.

Setting Goals

You must also set goals for the results of the changes after.

If you don’t have at least some idea of what you want to do with these results, you are flying blind.

To reiterate – don’t make assumptions:

Making assumptions is not a good idea at any part of the process.

The entire reason behind A/B testing is to make sure you have results from your tests to back up your changes before you make them.

To solidify an assumption into a concrete data point.

And that data point has convincing power.

A/B Testing Tools to Help Get the Job Done

A big part of A/B testing is working with user-friendly testing tools.

These tools will help you carry out your A/B tests with the utmost accuracy.

User testing is the most essential phase of usability testing.

Usability testing can be a critical part of the usability appraisal process.

User studying should be always a continuous process, instead of being conducted haphazardly.

The purpose of this testing is to produce a more systematic understanding of how people communicate with your site.

A usability evaluation really isn’t the same as a usability test.

To recognize whether you can find any difficulties with converting users on your site, you should conduct usability testing.

Once this is carried out, the screening procedure for user studies commences.

User studies can have many goals or a single objective.

As an example, user screening may be designed to:

  • Find out how users are really scanning your page.
  • Assess what’s actually grabbing their attention.
  • Address any deficiencies in your existing content.
  • Figure out where the weaknesses lie in getting them to click on that button.

Several tools exist that can help you out with this, including:

These tools are made for A/B testing and will help you define parameters you may want to test.

They will also help you carry out these tests in many cases.

The best thing about, unlike the other tools, is you get live feedback from people on video as they view your site in real-time.

Most People Don’t Do the Right Things When Conducting an A/B Test

Again, small websites with low traffic don’t need to A/B test.

Traffic volume is simply not enough to justify A/B testing.

At this stage, you should not be using A/B testing at any point in your process. It isn’t large enough.

Instead, work on interviewing your customers, finding out their pain points, and working on tweaking your site based on customer feedback.

Conversion rate optimization (CRO) is not the right thing to do for your site at this point in time.

A/B Testing Ideas You Should Try

In case you didn’t know, there are plenty of A/B testing ideas you can try to determine which ones will provide the desired results you are looking for.

But, there are some that are not widely well-known.

In fact, doing some of these can generate lifts in your conversions quite significantly.

You must have the courage to stray from your original idea when it’s wrong, and have the fortitude to explore different ways of communicating the same thing.

In that same vein, creating the same thing over and over from site to site is bound to get you into a dangerous rut that’s hard to crawl out of.

How Do You Know Which One to Do & Which One Not To?

It comes with experience and training.

The untrained CRO specialist will sometimes go after everything or elements that won’t have enough of a significant return on their time investment to do testing right.

A savvier, experienced CRO specialist will have common elements they test on most of their sites because they know that mistakes with these elements happen so often that their influence often causes a dramatic increase in production in just a few implementations.

Experience also comes with the decision-making process of choosing an element and making refinements based on this testing.

The following ideas are some A/B testing ideas you can try to see what changes to these elements can do.

Try Different Versions of Your Content

If you are unsure how to write content for your audience, perhaps it’s time to A/B test a few different versions of your content.

For example, if your audience is used to longer text and that is what they respond to – as is the case for many scholars and people in academia-focused fields, you may want to ensure your text reads like an essay.

But, for audiences who are more casual readers, you may want to test shorter lines of text, and shorter paragraphs.

Are people within your audience more or less those who require a technical guide that’s thousands of words long?

Test this variation also.

As someone who knows your industry, you are in the best position to make these judgments about what your audience may or may not expect from this testing.

Test a Larger Size Contact Form Button – or a Smaller One

Are you getting fewer people submitting your form than you would like?

Or perhaps you are getting the wrong information submitted.

This is where you can test distinct things about your form process, including your contact form button.

Tweaking the verbiage on your form to result in better conversions can result in a more accurate conversion process.

Did you know it’s recommended by some conversion experts to include an expectation of a timeframe for a response? Say “We will respond to your message within 24 – 48 hours” for example?

This is known as “setting your user’s expectations.”

Because when they are first reviewing your site, and they don’t have any idea who you are as a business, they won’t know the first thing about how long it takes for you to respond.

Such an element also promotes trust, because it tells them they will receive a guaranteed response rather than waiting for a reply that may fall into a black hole and never return.

Test Different Variations of Your Brand Messaging

How you communicate is just as important as how your content is interpreted by your audience.

No, they are not the same thing.

One involves you, and the other involves your audience.

You cannot know exactly what your audience will respond to without testing live variables.

You’re not psychic, and you don’t always know what will elicit the reaction you want.

It isn’t going to happen.

That’s why testing live variables and variations of these variables is important.

One such test you can carry out involves tweaking your messaging, especially when it comes to headlines, phrases in your content, and wording on your calls to action.

Perhaps your message is not quite as effective as you would want.

Or, your overall brand message needs tweaking.

Perhaps your brand tagline must be tweaked.

Maybe certain elements of your message are not executed properly.

One version of your message may be just right, leaving no doubt who you are as a company.

While a second version may fall flat and generate criticism on social media (which you also want to avoid).

You won’t know these answers unless you test these elements.

Psychic powers won’t help you in this arena.

Always Follow Your ABTs

When it comes to making changes on your site, always make sure that you follow the ABTs – always be testing.

They can reveal things about your users you may have never suspected.

As we have explored, by engaging in A/B testing on a regular basis, you can make changes that will produce dividends for your site.

Creating a solid testing methodology is the key to successful A/B testing.

Once you have planning worked out and final elements in place, all that’s left is the testing execution.

As I always like to say – always follow your ABTs – always be testing.

Your large project may depend on it.

More Resources:

Image Credits

Featured Image: Created by author, May 2020

VIP CONTRIBUTOR Brian Harnish Senior SEO Analyst at Bruce Clay, Inc.

Brian has been doing SEO since before it was called SEO, back in the days of 1998. Back then, SEO ...

A/B Testing: Why & How to Do Split Testing on Your Website

Subscribe To Our Newsletter.

Conquer your day with daily search marketing news.