SEO

The Problem with SEO: Mistaking Correlation for Causation

So you just saw a big jump in your natural search result rankings?  Congratulations, that’s great news!

But before you start patting yourself on the back for having all your hard SEO work pay off, it’s important to remember that just because your natural search rankings changed, that doesn’t necessarily mean that your results can be attributed to your SEO campaigns.

One mistake that many SEO workers (as well as the non-technical bosses they often report to) frequently make is to confuse causation with correlation.  We see an increase in SERPs rankings and assume that our activities are paying off, or we see a decrease in natural search performance and decry the “penalties” that have unfairly hit our sites.  Unfortunately, jumping to these conclusions is incredibly short-sighted, as it’s difficult to prove true causation between SEO campaigns and search results rankings.

To understand why this is the case, let’s start by reviewing the difference between “causation” and “correlation”:

  • Causation implies that a specific outcome was brought about as a direct result of a set of actions.
  • Correlation occurs when two or more variables appear to have a relationship, though no causative agent has been established.

To better understand this distinction, consider the following popular example:

Correlation 637x451 The Problem with SEO: Mistaking Correlation for Causation

As the graph above indicates, there is a proven correlation that exists between the amount of ice cream consumed and the number of drowning deaths that occur each year.  As more ice cream is consumed, more drowning deaths occur.

Now, does this mean that eating ice cream causes drowning?  Of course not!  In fact, a third variable – summer weather – is the causative agent in each case.  As the weather gets warmer, people tend to both eat more ice cream and swim more, which leads to the increase in drowning deaths simply because more people are swimming in the first place.

This same distinction needs to be applied to SEO results, as changes in your SERPs rankings could be occurring due to any of the following factors:

  • Your SEO Efforts – Any one specific element of your SEO strategy, including your link building techniques, on-page optimization metrics, or other activities, could be resulting in higher rankings for your site.
  • SEO Mistakes Made by Your Competitors – If your site competitors are employing gray-hat or black-hat SEO techniques, your movement up the natural search rankings could be attributed to their mistakes, not your own SEO activities.
  • Changes in the Search Engine Algorithms – The search engines are constantly changing and reevaluating different ranking factors, so an increase or decrease in your SERPs rankings could be occurring due to the reevaluation of your site against a new set of standards.
  • The Devaluation of Specific Link Types – As spam links continue to be weeded out of the search engine indices (as was recently the case for blog network links), the value that they improperly directed to the sites employing them is removed, resulting in a drop in rankings for those sites.
  • Any number of other variables…

Given the number of potential causative agents involved, directly attributing SERPs results to your SEO activities is nearly impossible, although you can still generate meaningful data by tracking certain metrics.  Here’s how to do it:

Step #1 – Document Everything You Do

At a minimum, you need to actively track all of the following variables for later comparison against changes in your SEO results:

Track Your Keywords

  • Create an Excel spreadsheet that lists every single keyword you’re actively targeting, along with the page you’re targeting the keyword on, any pages of yours that appear in the natural search results for these phrases, and the change in your top placement over time.
  • Create a separate spreadsheet that captures the same information for the Top 500 keywords that generate traffic for your site.

Track Your On-Page Changes

Whenever you make changes to the on-site optimization of any of your pages (for example, changing your title tags or adjusting the keyword density of your articles), make an annotation in Google Analytics that contains the details about the specific changes you made and the dates on which they occurred.

Track Your Links

  • Create another Excel spreadsheet to track the number and quality of the inbound links that are pointing back to your site. This information can be downloaded using Majestic SEO or Open Site Explorer.
  • In addition to tracking the source of each link pointing back at your website, make a note of the PageRank of the referring URL, the anchor text used in the link, and the root URL of the referring site. Changes in the distribution of these specific link metrics can result in SEO gains and losses.

Track Your Social Signals

As social signals have been implicated as new SEO ranking factors, create another spreadsheet to measure your progress in this area according to all of the following variables:

  • Number of social networking profile followers
  • Number of Facebook status updates and Twitter tweets
  • Number of profile “likes”
  • Number of URL mentions on social networking sites

For the most accurate picture of how your SEO campaigns are contributing to your SERPs performance, monitor your keywords and links results at least once a week and your social signal changes at least once a month.  For accuracy, create Google Analytics notations for your on-site changes as they occur.

Step #2 – Keep Track of Your Competitors’ SEO Changes

Since changes to your site’s SEO results can occur due to mistakes on your competitors’ end as well as from your own activities, you’ll also want to measure the changes that your competitors are making on their sites.

So in addition to tracking your own activities, keep an eye on the following competitor metrics:

  • Changes in the number, type, and quality of their inbound links – As an example, if you see your competitor sending out an abnormally large number of press releases at the same time that their site surpasses yours in the SERPs, you could use this correlation to reverse engineer their SEO strategies and apply them to your site.
  • The number of indexed pages on their sites
  • Any major on-site changes that demonstrate a shift in their SEO strategies

Yes, this is a time consuming process, especially if you’re tracking the results of multiple competitors.  However, it’s a necessary activity if you want to increase your odds of determining conclusively which variables are resulting in natural search performance improvements.

Step #3 – Compare Changes in Your Metrics with Changes in Your Rankings

Now, whenever you see a change in your SERPs rankings – whether positive or negative – you’ll be able to compare the timing of your search results movement with quantifiable metrics.

Again, you’ll never be able to say with 100% accuracy whether or not your actions or your competitors’ SEO activities resulted in your site’s movement through the SERPs.  However, by keeping an eye on these basic metrics, you’ll be able to better tune your SEO strategies to the activities you suspect are resulting in the biggest SERPs gains.

What did I miss?  What other variables are you tracking in order to measure the difference between correlation and causation when it comes to SEO?

Share your recommendations in the comments section below!

071c63b926025330e9468435427b9f37 64 The Problem with SEO: Mistaking Correlation for Causation
Sujan Patel is a passionate internet marketer and entrepreneur. Sujan has over 10 years of internet marketing experience and started the digital marketing agency Single Grain. Currently Sujan is the CMO at Bridge U.S. a company that makes the complex immigration process easy and affordable.

You Might Also Like

Comments are closed.

8 thoughts on “The Problem with SEO: Mistaking Correlation for Causation

  1. You also have to consider non-SEO actions that can impact your SEO. For instance, a heavy PR campaign might drive a lot of branded traffic in one month that drops off the next. A strong PPC campaign could be sending a lot of paid traffic to site that covers for any losses in organic. That’s why it’s so important to not silo your SEO reports.

    1. Nick,

      Great point. You have to look at everything you do, it’s not going to be easy to measure PR or offline efforts but it’ll help truly understand your site’s situation.

      I’m writing a follow up post on how to measure your marketing/offline/branding/pr efforts as it relates to SEO.

  2. Grammar Nazi alert: you probably meant to say ‘mistaking correlation for causation’. Yes, the order of those two words is very important.

    The way you’ve said it – ‘mistaking causation for correlation’ – suggests that the problem is that SEOs confuse an actual causal relationship for having just a vaguely correlated connection. And your article actually says the reverse: that SEOs confuse correlated metrics with direct causation. The latter is a correct presumption, the former however not so much. I don’t know any SEO that confuses a causation for a correlation.

    Other than that, spot on.

  3. Sujan, wow, this is a very detailed “day in the life of an SEO.” This is exactly the type of work that I expect an SEO to perform on a team. It also explains in part why SEO is a full time job. Great guidelines!

  4. Thank you for the great post. As documenting, keeping track of changes and researching competition is an essential part of SEO Thanks once again for describing the difference between “causation” and “correlation”