shutterstock_6792991
Google

Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

The past two years have seen vast changes in the SEO industry. The changes, though mostly positive, included the aggressive activation of Google penalties, whether algorithmic or manual. As a result, numerous websites ended up with a manual penalty or worse, an algorithmic penalty or a filter. Unlike algorithmic penalties when we’re at the mercy of Google’s algorithm refresh mechanism, manual penalties can at least be revoked following a lengthy process of correction that culminates in a reconsideration request. Or can they?

Before we closely examine the necessity of this thorough procedure using real life examples, I’d like to address a few key terms to ensure we’re all on the same page.

Penalty:

  • Can be manual or algorithmic
  • Can be time limited or not
  • Can sometimes be removed by filing for reconsideration

Filter:

  • Solely algorithmic
  • Most common (I estimate most website owners aren’t aware they’re dealing with a filter)
  • Cannot be removed by filing for reconsideration

Banning:

  • An extreme measure against heavy spammers
  • Often irrevocable, best to desert site

Many people confuse a Penguin penalty with a Penguin filter, for instance. The difference there is that a website that receives a Penguin penalty will likely experience decreased rankings in numerous keywords and drop dozens of page rankings – something any site would struggle with. Conversely, if a site is faced with a Penguin filter, it will experience a mild decrease in rankings, generally single digits, and only for select keywords. Banning, on the other hand, is an extreme measure not taken lightly by Google. Personally, I haven’t heard of many websites that were banned altogether from search results, unless they were using explicit black hat methods and other practices that blatantly violate Google’s guidelines. Other notorious past penalties are the Sandbox and -950.

Typically, filters are applied in instances where duplicate content, slow loading time and inappropriate use of tags are found. If you’re interested in learning more about filters, refer to the full list in an old-but-great post written by Barry Schwartz.

We can only hope to remove algorithmic penalties and filters once we’ve resolved the problem and an algorithm refresh takes place. I’d like to focus on manual penalties since so much has been written about tweaking an optimal procedure that will successfully remove it. We’ve seen everyone weigh in on the topic, from superficial articles to thorough, professional guides offering a step-by-step manual to handling a manual penalty efficiently.

A number of years ago, seeing the penalty removal business flourish, I too saw the value in learning the ropes. I started to delve into the subject and after months of research and consultation with experts worldwide managed to formulate an efficient method for manual penalty removal that has enjoyed very high success rates. The system is tedious and mostly manual, but it pays off eventually. We have yet to file more than three reconsideration requests for clients before removing a manual penalty, and we’ve handled over 120 cases so far.

Is Active Link Removal Necessary?

There is one case I’d like to use as an example here, since its journey to penalty removal was unique and somewhat strange – making me question the necessity for the strict guidelines Google set forth for penalty removal.

It all started when a new client approached us about removing a manual penalty. With 15 years experience and 12 years online, his company was well established and enjoyed a positive reputation. This company collaborated with NASA, the White House, the Department of Homeland Security and other large organizations, leaving no room for doubt: we were dealing with a legitimate and honorable company.

Examine Links

After carefully inspecting the client’s website and link profile, examining a his link profile, the link profiles of the sites that linked to the client’s website, and of course analyzing the competitors, we concluded that were weren’t dealing with an overly complicated case. The website had thousands of incoming links from thousands of different domains, some high quality and some of lesser quality as expected for a website that’s been around for more than a decade. The bad links were weeded out and found not to comprise a high percentage of the overall links, making us even more confident of being able to remove this client’s penalty relatively swiftly.

The first things we did after the research outlined above, was map the existing incoming links using all available sources: Google Webmaster Tools, MajesticSEO, Ahrefs, Open Site Explorer, and Yandex Webmaster Tools. Secondly, we ran all the links through two systems to ensure they were all live. All the links that were marked as non-existent (404 errors, link not found, site is down etc.) were tagged and forwarded for manual inspection by our team, to ensure they were in fact gone. In our experience, sometimes live links and live pages can be marked as non-existent due to heavy traffic, server issues, time-out responses, and other problems. To be thorough, we always conduct a manual verification.

We opened a Google Docs file and copied the links. The reason we managed this using Google Docs instead of MS Excel, for example, is outlined in this video by Matt Cutts. Then, it was time for the longest and most rigorous phase – manually sorting the links. We sorted through all the links, deciding which to keep and which to remove. For every link we decided to remove, we wrote a reason and tagged it in red.

13 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

Our team manually pulled the contact details of the website owners using three different methods:

1)      Listed on the page itself

2)      Using WhoIs

3)      Contact page found on the site

Outreach

After sifting through the links and obtaining the contact information, we moved to the outreach phase. This stage is difficult since Google won’t allow sending a large number of e-mails with the same exact wording. Those who try often deal with having their IP address flagged and blocked, preventing the delivery of identically worded e-mails in the future. Though I understand theoretically, this prevents people from sending mass e-mails and spamming the web, I feel that it greatly hinders websites owner’s ability to address penalties efficiently by contacting problematic websites.

To solve this problem, we searched for a different e-mailing platform other than Gmail, which somewhat complicates the process but is better than the alternative. We manually sent a personal e-mail to each and every one of the websites flagged in three stages, asking them to remove trouble links. Simultaneously, we submitted contact forms on all websites that offered them, with similar requests. During the documentation phase, we focused on grabbing screenshots of forms submitted and gathering copies of all the e-mails we sent.

Document Progress & Submit for Reconsideration

The process was documented further on Google Docs, including the link owners’ contact information, the decision whether to remove or not to remove, the e-mail sending dates, the sent e-mails’ source code, screenshots and form submission dates, payment details (yep, some people ask for money to remove a problematic link), and the final status of those who removed the links, refused to remove the links, and never answered. At the culmination of this process, we prepared a disavow file that we then submitted to Google, and waited at least 48 hours, precisely as Matt Cutts instructed, before filing a reconsideration request.

Resubmit

The first request was denied. We weren’t phased since we knew this was a possibility, especially on the first try. However, Google’s answer was different in that they gave us examples of bad links – which was great in itself. We took a look at the file we prepared and couldn’t find those links anywhere among those we were able to pull from our resources. Refusing to despair, we ran another scan to find links from all the biggest sources, and found an additional 40 referring domains. We added those to our list, and embarked on the process once more, up until filing for a reconsideration request once more. I feel it’s necessary to emphasize that John Muller stated numerous times that it’s often enough to pull links only from WMT, and only in cases of severe spam would other tools be needed. Here’s a video of the Google Hangout where John Muller says this explicitly (video starts at relevant section). Either way, we filed for reconsideration a second time.

22 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

Unfortunately, our second attempt was also met with a denial, once more giving us two examples of bad links. This time, one had appeared on our list and was removed in the first request for reconsideration, and the other one was not on our list but was found to be borderline (in terms of spam). This is odd in itself, since Google representatives tend to give indisputable examples of links that breach their guidelines.

33 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

Once more, let me emphasize that the link itself wasn’t that bad. Truthfully, three years ago it would have met Google’s guidelines fully, and it was actually a four year old link. For some reason, it did not appear on our list. Since we knew the link was currently being questioned, we re-ran the scan and tended to it in addition to other links similar to it. What angered us more was the link that had appeared on our list and which had been removed on the first reconsideration request.

We re-checked the page over and over again, including the code, using four different software and four different team members in order to verify that there was no link still present. Verdict: the link had been removed as we thought. Why would Google send me an example of a bad link which I was removed a month and a half earlier, which I reported? I wrote a letter asking them to explain where they were pulling these links from since I could not be expected to handle links I can’t see, and in addition I asked them to explain how they can see a link that no longer exists. Naturally, I received a generic response stating that they were sorry, but the penalty was not removed – without offering any human response to my concrete claims.

Re-Reviewing Links

We went back to square one and reviewed all the links, sifting as though we hadn’t already done it twice. We wanted to verify we hadn’t let a few bad links slip under our radar, and so we hardened our criteria and wrote off even more websites. We submitted a request for reconsideration and for the third time, and were denied. This time, the bad link examples were of the no-follow variety. This is again questionable since on a number of occasions, John Muller stated that no-follow links need not be removed. Here’s a direct quote from one of his recent interviews:

44 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

Needless to say, we were starting to despair. It seemed that no matter what we did and how thorough we were, we received contradictory answers that were irrelevant to our process. The information coming from Google was also apparent on the WMT, as evidenced below:

54 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

While they explicitly write that no action is required on our part if bad links were created outside of our control, they recommend that we remove these links and file for reconsideration. Confused? Let me spell it out for you: without removing the links and filing for reconsideration, the penalty would remain – and action is most certainly required on our part when it comes to unnatural links whether they were the result of our activities or whether they stem from competitors’ malicious attempts.

Fresh Approach

At this point, we realized we were in a never-ending cycle and needed to change direction. Since we know that the links provided through WMT are just samples, we decided to scan the web using all our tools every two days, to compile an updated list. Moreover, we halted all link building efforts from the dawn of the process so that new links were not added to our link profile. After about two weeks we managed to collect another few dozen domains and handled them as described above. We decided that as an additional step, we would physically remove a number of pages from the website, which Google pointed as being linked to from bad sources. John Muller has said in the past that in the event that we encounter a problem and want to get rid of the links referring to the page, we can simply transfer the page to a new address – returning a 404 error and eventually making Google disavow those incoming links. We took it one step further and just deleted the page, without even opening a replacement. Here’s what John Muller stated:

63 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

We made an updated file that contained all the links found from the first stage, made a newly updated disavow file and filed for reconsideration. In this new request, we mentioned everything that had happened, explained the process in detail and emphasized that this was a completely legitimate and established website that cooperates with some of the largest bodies in the United States. As usual, we attached all files.

Denied…Again

This time, the request was denied sooner than we expected, within just a few days. No examples were attached, nor any explanation given. I can’t even describe to you our frustration at that point, let alone the site owner’s who had given up all hope – questioning Google’s professionalism and reliability. We decided to stop all activities and scheduled a meeting to discuss the situation in one month. When we held our meeting, the site’s owner wanted to write Google a personal message, using the reconsideration request route as a direct communication channel. He did not attach any files or documentation. We were opposed to this, but he was adamant, so he started to type: “Hello,”… and then, he accidentally hit “Send”.

I asked him why he checked the check mark before he started to write the e-mail and he apologized, saying it was an old habit. We didn’t revise our message, and what do you know, four days later we received a message saying the penalty had been revoked! Just like that. No further steps were taken, no documentation provided, not even a fully written letter. Despite our vast experience in the field of penalties, and despite our impressive success record, we had never stumbled upon anything like this.

71 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Screenshot taken 5/4/2014

If you carefully examine the process we went through step by step, you can see that Google’s reported ‘procedure’ for penalty removal does not align with the way Google handles reconsideration requests. Does anyone even check these or is this a flawed automated service?

Around this time, we heard from our colleague in Chicago, who told us he never removed any links. He simply sifted through link lists and entered bad links into a disavow file. No documentation of communication with any webmasters whatsoever, and files. According to him, this method has enabled him to lift penalties from over 30 of his clients’ websites. Sounds a lot easier, doesn’t it?

We made a business decision early on to thoroughly address issues that caused penalties to be issued to our clients, and therefore did not follow in our colleague’s footsteps. Moreover, we took some of his descriptions with a grain of salt, since every professional manual out there outlines a thorough, meticulous manual process similar to ours. There may be some rounded corners here and there, but nowhere near the superficial approach he displayed.

Surprisingly, a few days ago I stumbled upon this post, describing a very similar fuss-free process for penalty removal: no active link removal, simply using the disavow function. This writer claims to have helped over 60 clients successfully have their penalties removed!

Now, we began to wonder if there was something to this much simpler way of going about penalty removal. Was Google effectively pushing an extreme route to ensure that penalized websites would undertake a lengthy manual procedure that would ‘clean up’ the internet? Perhaps none of it was ever truly necessary.

Everyone here follows Google’s publications, Matt Cutts’ videos, and John Muller’s interviews – all stressing the importance of active link removal and the ineffectiveness of submitting disavow files without said link removal. However, we’re hearing more and more accounts of instances where these statements were blatantly contradicted.

Undoubtedly, the penalties issued and the subsequent penalty removal efforts made by countless websites have helped to clean the internet considerably – but is Google using SEO as its own personal clean-up crew for the internet?

What’s been your experience with the ‘traditional’ penalty-removal process, and have you attempted to skip active link removal? Share your experience in the comments.

Featured image via Shutterstock

 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

Ben Oren

Director of Web Marketing at Whiteweb LTD
Ben Oren is the Head Marketing Consultant at Dynamic Search™ - a US based, reputable web marketing agency handling small and medium clients worldwide. Ben specializes in handling web marketing efforts and boosting online conversion for large corporations in highly competitive niches, mostly in the US and Europe. He is also the Director of Web Marketing at WhiteWeb technologies.
 Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

You Might Also Like

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

25 thoughts on “Is Google Unfairly Using the SEO Community to ‘Clean Up’ The Internet?

  1. Thank your very much for the article and the in-depht info!

    Well, i can only speak for some (not all) SEO agencys in Germany: we are using the disavow-feature in WMT from the beginning and never contacted a webmaster for a link-removal.
    Reason 1: sometimes there are about +1000 sites to contact.
    Reason 2: i can not see the logic behind the line of argument for link removal: first contact webmasters, and then disavow? Google says, with the disavow feature, you set these links to nofollow in the eyes of google. All perfect with that, i say.

    Greets

    Tilmann

    1. Thank you for your kind words, Tilmann. In the past, we’ve had penalties removed relatively quickly and we suspected that there’s no need to contact webmasters. However, we never wanted to risk it by testing this and possibly harming clients’ detox process. The case I described, along with numerous accounts such as yours, strongly suggest that the reaching out process is unnecessary.

  2. Hi Ben! Great post! I’ve had a similar case… I did the whole link audit process, documentation, contacting webmasters etc. and submitted a reconsideration request for a manual action penalty (unnatural links). Google denied and said it didn’t work, they gave us 2 example links … we then deleted all other links that could be bad… but even though Google sent us a message that our request is denied, now we can’t file a request anymore because it says “no manual actions” in webmaster tools, the function just disappeared. Of course, we still don’t appear in Google for our own brand term. So now we are supposedly in a manual action, but it doesn’t appear in webmaster tools. No logic what so ever…

    1. Thanks for sharing this peculiar case, Stefan. In my opinion, it could be that your site is under an algorithmic penalty (Penguin). Though the case you’re describing is unusual, it’s not the first I hear of websites having manual penalties removed without receiving a notification about it in WMT (by error). The reason I believe your site received an algorithmic penalty is that mostly, manual penalties and Penguin arise from the same circumstances, and go hand in hand. However, when thorough detox is performed and the manual penalty is removed, in my experience, Penguin is also lifted (in the next algorithm refresh). It may be that the manual penalty you received was lifted and you never received a notification about it, and the Penguin penalty was also removed – which you’ll only see on the next refresh. Hope this helps.

  3. Thanks for sharing! We recently had a client that came to us with a fresh penalty from another seo company. We warned them that it would take some time, and learned to our dismay that the former seo company had already done a disavow and a reconsideration request. We were furious as there was no way they could clean up such a toxic link profile in a few days. Upon reviewing the documentation, there was almost no time given for webmasters to respond and take down links before the reconsideration request was. To our surprise, Google lifted the penalty within 7 days anyways….

    1. Thanks for the comment. Guillermo, this type of experience strongly supports the hypothesis that in most cases, the manual penalty is only given as a general scolding, and it suffices to show only disavows. I assume that in severe cases of spam link-building, Google is more strict. Ultimately, we need to remember that this all boils down to actual people performing manual reviews, and there may be stricter and more lenient reviewers – leaving us to question Google’s internal guidelines in these cases.

  4. Interesting post. I have had several cases like this that were extremely frustrating because Google kept coming back with example links that were extremely difficult and often impossible to find. I have found that the sites where Google does this type of thing are ones that have had years and years of spamming history. I’m not saying that I agree with Google making it harder for these sites, but I’m saying that in some cases when there is a long standing history of spam, they want to really hammer in the point that creating unnatural links is not a good idea.

    In regards to sending the near blank reconsideration request and then passing, that gave me a little chuckle. I see so many webmasters who spend hours fretting over getting the wording correct in their reconsideration request. I usually tell them that the wording is not important, but that the cleanup work that you have done is the most important thing. I too have had cases where we did not remove any additional links but replied with something like, “We have done all that we can. There are no more links for us to remove” and we have passed. I believe that what happens in these cases is that the webspam team chose a more favorable sample of links to review and it just happened that the majority of the links that they chose to review this time were ones that we had removed. It’s also possible that some of your bad links died out and that put you across the threshold of how many links they wanted to see removed.

    1. Hi Marie, hope you’re well. Thanks for taking the time to read my article, and for responding. I agree with you that Google may be harder on sites that have a vast history of spam. The site I talked about has existed for 10 years, but has never engaged in pure spam – most of its links were, at the time, completely legitimate. Some of them may have been borderline, but none of them were 100% spam. This leads me to believe that there is a variation in the leniency exhibited by members of the spam team: some are just more strict, causing the results to be inconsistent.

  5. I agree with Tilmann It is possible that you don’t need to contact the webmaster to get links removed you can just use disavow. Think about it if Google gets enough disavow request that concern a certain website Google will simply just ignore all future links from that site anyway.

    I have about 25 cases where I just used the disavow tool and had penalties lifted. The thing is you really have to dig deep into the site to get all the links and not just use WMT.

    1. Hi Vinny, you’re absolutely right – I agree with both of you and that’s precisely the point I tried to make in the article. However, Google’s official declarations contradict this claim. As I’ve shown, Matt Cutts has said that only using disavow wouldn’t work, and we see cases like the one you described, and many more, that attest to the opposite. This raises the question I started out with – is Google (unfairly) using web marketing professionals to ‘clean’ the internet of spam links? It would appear that by insisting on the lengthy process of reaching out to web masters to remove links, it is doing precisely that.

  6. This is an interesting case study by SEJ. I’ve been also watchful that no one really interprets the reconsideration request letters, and this is the entire some detailed simulation, for some moment.

  7. I decided to test the easy route with one of our clients last Thursday. No outreach, no documentation, just a simple copy-paste into a text file to disavow.

    To my surprise, the manual penalty was gone on Monday morning. I didn’t even have to submit a reconsideration request.

    Might be a fluke or a coincidence, but I’m not complaining.

  8. Your problem is here: “Moreover, we halted all link building efforts”

    You don’t understand the core concept behind this penalty: Google doesn’t like any link building. No link building. Remove the words “link building” from your vocabulary. You were building links through half of the process that penalizes you for building links and you were surprised they don’t lift the penalty?

    Just like your friend we only had cases of submitting disavow file (but with real hard cut of ALL unnatural links) and penalty gets revoked from the first reconsideration request.

    People who fail to get the penalty lifted from the first try usually don’t understand what disavow ALL of the unnatural links means. It means all of them. No, your awesome guest blog post with non commercial keyword is not a natural link, disavow it. No, your blog created on blogger / wordpress / your own domain solely for the purpose of linking to that page even though you wrote a few more posts on it and / or linked somewhere else as well to make it look more natural is not a natural link, disavow it. No, your link from a human edited web directory that no one ever visited is not a natural link, disavow it.

    It’s so simple: If you knew you’re getting that link before you got it, it’s not a natural link, disavow it. And please, stop link building.

    1. Hi Tony,

      Thanks for your thorough response.
      I disagree with you. You stated the root of the problem I allegedly have by saying I don’t understand the meaning of link building and that I need to cease it immediately. I’d like to shed a bit more light on the story and get your opinion. When I said we stopped link building, I never said we built links beforehand. We never performed SEO for said company, they only approached us after they received a penalty. We stopped link building activities carried out by them previously for two reasons: firstly, the links were of low quality; secondly, we needed to stop the inflow of links until we solved the problem.
      I don’t feel it’s helpful to make declarations such as “stop building links immediately” without having evidence to back it up. Link building (or link earning as it’s called today), when done right, is the single strongest way to leverage a site in search engine rankings. This isn’t my private opinion, but a long-stated fact backed by numerous statements, even from Matt Cutts. There’s no other equally effective way to promote sites than incoming links. Two months ago, Matt Cutts said in a video that Google was working on finding a different solution, but currently, every link-free algorithm they’ve ran has yielded less relevant results than the current link-based one. According to him, links will stay central for numerous years.
      I agree with you that not every link is valuable and serious thought needs to be given to incoming links, but what you’re describing is not realistic. I’d love to hear from you about your contingency plan, or even if you could show me one website that’s ranked high in search results not as a result of links. If you adopt fishy approaches just to get links then obviously there’s a problem, but if you do so in a high quality way, I do not feel there’s a problem.
      Take, for instance, this post. Every outbound link goes exactly where it should. These are not links to my clients, these are links which take readers exactly where they expect to end up – is that spam? Clearly not, these are Google’s guidelines followed to the letter. It’s important for me to clarify that every marketing activity we engage in today comes down to one main goal – getting links. No matter how you call it, at the end of the day, when we share our posts, tweet case studies etc, we expect people to link to us as a result.
      In conclusion, different marketing actions have different advantages and objectives, but the highest ROI is consistently yielded by organic results and for that you need to rank high.

    2. This is what I got from your comment, you even end it up with “And please, stop link building.”
      Anyway, as long as we both agree that links are still important, as long as they are with value, we’re good :)

    3. Toni, Your answer seems a bit contradictory considering the fact that it’s coming from someone with over 15k site-wide links with the anchor text, “Developed by Inchoo” and not to mention all of the .in and ro links you have that are obviously spam. Does this seem natural to you?

      The misinterpretation of link building caused people like you (no offense) to step back from what matters the most. Is a link from BBB, Yahoo directory, Chamber of Commerce, and contributing to the local community considered bad linking to you?

      I’m not afraid of the words SEO and link building because I’m not chasing the algorithm and I’m building connections to stay in-front of my potential prospects. While you suggested to “remove the words link building from your vocabulary” and then say “I didn’t say you don’t need links”, I personally suggest you take a closer look at what you do first that is outside Google guidelines and very easy to discover both on and off your website…

  9. That is a really messed up story, man. Sorry to hear about it.

    Yes, Google has turned webmasters into a backlink policing force. I never would have imagined it 3 years ago.

    3 years ago you just had to get dofollow high PR links (contextual if possible) and you were done.

    Now you have to do the same, but make sure to get them on respectable niche related sites…..plus find and report/remove any links that may be naturally or unnaturally made to your site.

    Its an (unnecessary?) added expense and service that Google has forced on the SEO/Internet community.

  10. The reason Google wants you to (try to) remove the links first is for a form of sadism, to punish you for placing those links in first place.
    If you didn’t put them (and you did, maybe your client did…), don’t bother. Just disavow, disavow a lot, disavow often, disavow brutally.

    1. Stefano,
      I agree with you in that point. Google want us to suffer during the process in order to learn a lesson.

      Please don’t forget that disavowing weakening the website and a lot of links that back then were legit, suddenly considered spam and Google punish us for that. This is unfair in a different way.

  11. Actually Google aren’t having us clean up the internet, they’re having our clients do it (our clients, who foot the bills). Large companies are employing (often multiple) SEO agencies to clean up their ‘real-estate’ online.

    I don’t think we need to complain – but our clients might. They’re paying for all of Google’s well intentioned manoeuvres directly.