The expert’s guide to SEO A/B testing

James Bentham

Head of SEO


SEO

SEO is not an exact science; there are no hard-and-fast rules for increasing impressions, clicks, or conversions. That’s why methods such as A/B testing are vital for helping SEO teams make effective decisions about carrying out changes to web pages.

For this article, our SEO team has come together to explain what A/B testing is, the different kinds of SEO landing page tests, how to conduct them, and the beneficial evidence SEO A/B tests can provide that lead to improved SEO results.

What is A/B Testing?

 

A/B testing is a method that compares two versions of a webpage, app, or other digital asset to determine which performs better. The two versions (A and B) are shown to different user groups, and user behaviour is analysed to see which version leads to higher engagement, conversions, or other desired outcomes. The results help businesses alter the tested medium more effectively, leading to an end goal of achieving more beneficial results.

The differences between SEO and CRO A/B tests

 

SEO and CRO are prime candidates for A/B testing as each requires trial and error to determine the best approach. However, CRO and SEO A/B test methods are very different.

 

CRO A/B testing

 

CRO refers to optimising a site for the users to increase conversions across visitors. Conversions can be anything from signing up for a mailing list to completing a purchase.

In CRO A/B testing, half of the visitors to a page are served one variant of the page (A) while the other half of users are served the other variant (B). Based on how users respond to those two variations, marketers can determine which is the better approach.

SEO A/B testing

 

SEO split testing groups pages into control or variant. Unlike CRO A/B tests, in SEO A/B tests, only one version of each page exists. Once the testing stage is complete, SEO experts can use the data as evidence to inform decisions that will improve web page results.

SEO A/B testing is an excellent way to determine how much a change will impact web page rankings. It aims to highlight where technical changes can be made to a website to improve traffic and positions in SERPs.

How does an SEO A/B test work?

 

SEO split tests work by creating a test and control bucket. SEO A/B tests take a family of landing pages (e.g., ‘shoes’) and change half of the pages, such as trainers, to create a variant. Then, the remaining half of the pages, such as heels, are kept the same, known as the control.

From here, the A/B test means every visitor sees the same version of the control (heels) and the variant (trainers) pages as everyone else. However, the two pages will be visibly different. This approach avoids creating duplicate pages, which can negatively impact the page’s SERP rankings and affect the A/B test results.

Splitting identical groups of landing pages, such as shoes, into a control or variant isolates the positive and negative results, allowing web pages to be updated quickly and efficiently. Plus, changes can be forecast to see if the variant group significantly outperforms the control group.

 

 

How to run an SEO A/B test

 

The steps to carry out an SEO split test are:

  1. Identify the group of pages you want to improve the performance of
  2. Select the tests or changes you want to run
  3. Forecast the results seen as part of the SEO A/B test
  4. Group the pages into either the control or variant
  5. Make amends to all pages in the variant group
  6. Begin running the SEO landing page test
  7. Compare the metrics from the control and variant groups.

 

After the SEO landing page test period ends, it is time to assess the results. This is where the forecasted traffic is compared to the actual traffic of the control and variant groups. If the variant group outperforms its forecasted traffic and the control pages don’t, the changes have statistically improved performance and can be implemented across the entire website.

Challenges associated with SEO A/B testing

 

SEO A/B testing is great, in theory. However, before conducting it, specific issues need to be considered. If a website’s traffic is incredibly low, SEO A/B testing can take weeks, sometimes months, to reach statistical significance, which isn’t ideal if changes need to be made quickly. On the flip side, if a website receives vast amounts of traffic, conducting SEO A/B testing can lead to significant dips in performance as web pages are split into two different versions.

Also, depending on how a website is built, SEO split testing can heavily rely on a web developer’s time to implement technical SEO changes. Therefore, dedicating portions of a developer’s time to make changes can prove costly if an SEO A/B test is unsuccessful. However, SEO experts can also use a content delivery network (CDN) to carry out landing page tests.

CDNs help to speed up website content load times for users regardless of geographical location. The CDN comprises point of presence (POP) locations and edge servers, with multiple edge servers located in various POPs. CDNs store a cached copy of websites, so when a user visits a site, their browser requests the website copy from the nearest edge server. This delivers content much quicker and speeds up load times.

 

Using a CDN for SEO A/B Testing

 

There is a small margin between when the CDN sends a webpage and when the user’s browser receives it. During this time, SEO specialists can inject small snippets of code that alter the website as it appears to the user. By SEO teams injecting this code, it enables experts to test changes on the variant group of pages without making any actual changes to the pages themselves. But, if any adverse effects do occur, the process can be stopped without relying on a web developer to revert pages to their previous state.

How to conduct an SEO A/B test with a CDN

 

The steps for running an SEO landing page test using a CDN are as follows:

  1. Identify the changes that need to be made
  2. Select like for like pages and group them into control and variant
  3. Forecast traffic changes for each group and identify the period the tests will be measured against
  4. Implement changes using a CDN
  5. Monitor traffic over the pre-determined test period.

 

If the CDN SEO test doesn’t show any real success, all that needs to be done is to delete the code from the CDN. However, if the CDN SEO A/B test succeeds, it gives SEO specialists quantitative data to support getting a website developer involved to make changes across the entire website.

Challenges related to CDN testing

 

Just like a standard SEO A/B test, there can still be obstacles to using a CDN for test purposes. Firstly, only small amounts of code can be injected into the CDN. If the code is too large, it can significantly affect page load times and impact test results.

Another challenge is that the page groups (control and variant) being tested do not contain identical content, even if the structure is the same. This means forecasting traffic for both sites and benchmarking changes against each individual group rather than comparing traffic directly between the two groups.

Thirdly, CDN testing is only as successful as the amount of traffic visiting a website. Seeing any results can take a long time, if there’s little traffic, or conversely, if you get lots of traffic, CDN A/B tests can affect overall SEO metrics.

 

As I said at the start, SEO is not an exact science. However, using methods like landing page A/B testing and user data overrides guesswork, which means SEO specialists can make effective changes that improve overall performance and deliver successful results.

Are your landing pages not performing as desired?Get in touch and we'll conduct SEO A/B tests for you