Many website, businesses, and employees have been affected by Google’s recent clamp down on spammy and manipulative link practices. It is becoming an all too common problem for small business, and we are increasingly finding that new clients have been affected by penalization and need help finding a resolution.
Whether a site has been hit by a manual penalty, or has suffered a drop in traffic following a Penguin update, there is always a lot of work required in order to get a penalty lifted. Many of the tools needed are expensive, and many of the popular methods require a lot of manual work. This post offers some tools, guidelines, and methods for reducing the time and cost of penalty removal work.
The first step to any kind of penalty recovery is figuring out if you have a penalty and what kind of penalty it is. With manual penalties this is nice and easy as Google just tell you in Google Webmaster Tools. Algorithmic penalties are a bit more tricky, and normally you have to break the old correlation ≠ causation rule when deciding that one is present.
When dealing with new clients, we try to qualify them as quickly as possible – often clients don’t even know that they’ve been hit by a penalty. A couple of tools can give you a quick indicator:
Website Penalty Indicator – Cost $0
My company developed this tool recently (built using SEMrush traffic data), which can be used for a quick visual check on traffic trends, and if a website has seen any large drops following Google update announcements (or not, as the case may be…)
ahrefs – Cost $0 (paid plans available)
Our backlink tool of choice, ahrefs is largely considered among the SEO community to have the largest and most accurate index. For our budget-busting analysis we’ll be making use of the free account, which gives you restricted data and only 3 lookups a day – but that is perfectly adequate for our needs at this stage.
By looking at a site’s backlink numbers and their anchor text ratios you can get a pretty good idea how manipulative their backlink profile is, and whether it might be worth doing some further investigation.
Screenshots from these tools along with a brief explanation should be enough to convince most prospective clients that it is worth doing some more detailed investigations, giving you deeper insight into their situation.
Panguin – Cost $0
This a fantastic tool for identifying penalties, but it does require you to have access to the Google Analytics account of the site in question, so it is not much use for pre-qualifying. However once you do have GA access, it is incredibly useful. Once you log in, you choose one of your Profiles (Views) and Panguin will overlay Google update data over your Google organic search traffic.
You are looking for an update followed by a fall in search traffic (or a series thereof), which could imply a penalty is present. Make sure to compare the data to previous years to ensure it isn’t simply a seasonal fluctuation.
SEMrush – $0 (paid plans available)
A sublime tool for competitor research, SEMrush makes use of its database of ranking history for millions of keywords to give you loads of valuable data. In the absence of keyword ranking history, the SEMrush overview (free) can give us a great idea of previous trends.
Combining all these sources you should be able to make a fairly accurate judgment as to whether a site has been penalized or not.
Total spend so far – $0.
Identifying Unnatural Links
For the purpose of this post, I’m ignoring Panda type penalties as that is often about improving site content itself, and I want to focus on link-based penalties. Often the biggest job when working on a penalty case is identifying the links themselves. Sure, you can use premium services like Link Risk and Link Detox, which can help a lot with the identification work – but we are on a budget here, people!
The first step in any link cleanup is to actually collect as many links as possible – and more data is always better.
Google Webmaster Tools – $0
Although Google have stated it is sufficient to only use the data listed in Google Webmaster Tools, there are two problems with this:
- GWT lists different sample links at different times
- You still need more link intelligence (e.g. anchor text) to make any kind of decision about link quality
But as a starting point, you can do a lot worse than GWT link data. If you want to be really thorough, log into GWT every day over a couple of weeks and grab the data every day, then collate and dedupe in Excel.
Bing Webmaster Tools – $0
Just because they don’t send you any traffic doesn’t mean you should forget about Bing! Their inbound link data gives you links and anchor text, so it can actually be more useful than GWT.
Fiverr – $5
I know I’ve been spoiling you with all these freebies, but if you can’t afford $5 (£2.99 to us Brits on today’s exchange rate) then you’re probably not charging enough for your valuable service. Why Fiverr? ‘Cause you can make use of someone else’s costly tool subscriptions and just pay them for the link data – one ‘gig’ can get you backlink data from Majestic, ahrefs, AND Open Site Explorer. Best 5 bucks you’ve ever spent.
With all that lot, you probably have enough link data to get going with link classifications. Get all your data in Excel and homogenize the columns as much as possible.
Before you spend any time even looking at your links, it is worth checking if the links are still live. Link checking tools will only stay up-to-date to a point, and sites that have been the target of spam are getting pulled down all the time.
Scrapebox Free Link Checker – $0
Dedupe your linking data and plug them in to Scrapebox’s free incarnation of their popular software, which will go and check if the links are still live in the first place. For any that aren’t, double-check that no other links from that domain are live, then move these across into a spreadsheet or workbook called ‘Links Removed’. If you are filing a reconsideration request for a manual action, you might as well add these links to your list of links that you managed to get removed and make it look like your work.
Once you have a final list of links that you know are still live, remove duplicates by domain, as we will only be checking links at the domain level (when was the last time you saw a ‘good’ link on a crappy site?).
You Don’t Need To Manually Check Every Link
I’ve heard many times that during the link classification process you ‘should’ check every single domain manually to make sure you don’t accidentally mis-classify good quality links.
I call BS on this.
The reason we have unnatural link penalties in the first place is because SEOs took normal legitimate linking practices and spammed the crap out of them. And how do you scale things quickly? You look for footprints, find one that works – rinse and repeat.
We can very quickly speed up the link classification process by finding ways to reverse engineer these footprints and bulk classify lots of links at once.
Excel Custom Filters
Working on Excel, we’ll use custom filters to try to identify some footprints and remove the amount of manual classification required. Go to Data->Filter and filter across your entire data set, then select the Domain or URL column and choose Text Filters->Contains
This will likely return you with all or most of the directory submissions for this site. This doesn’t mean that they are all necessarily bad, ‘SEO directories’, so cross-referencing against something like anchor text usually helps you make a quick decision on the value/quality of each link. If you manually check a few you’ll quickly see the link builder’s footprint, and you should be able to classify the sites without ever visiting them. I tend to cut and paste links out into new workbooks, so I end up with a ‘Good’ list and a ‘Bad’ list (for future removal/disavow work, the list of Good links is your best friend).
Then apply this same principle to a range of different search words:
Once you spot a particular tactic has been used, try out different variations of this method to see if you can quickly identify any more links.
Rank Cracker – £0
This software is actually designed as a link building tool, but we will use it to do the opposite. This tool runs through a list of links and determines which automated link building software can be used to replicate them. In our case – which automated link building tool may have been used to create the links in the first place.
Again, don’t use this signal alone – make sure you double-check the results or try to pair it with anchor text or something – but it can be a good indicator that the links are manipulative.
Another cool feature of Rank Cracker is that it provides you with contact details for the sites on links it could not identify, which will be useful for link removal work (see below).
Some Manual Checking Is Unavoidable
Once you have exhausted this footprint method, you should have identified a decent amount of links, thus reducing the amount of (unavoidable) manual work required. However, even when you are working through a list manually you can take some shortcuts. Filter by anchor text and look for money keywords – it is highly likely that these links are manipulative.
You can also – after a while – begin to judge a link based on its URL alone. If the URL or the domain itself does not look particularly natural, it is unlikely that the link is. Again, you will start to see patterns in the sites and URLs that can lead to further filtering – I have classified thousands of links using this method.
Total spend so far – $5.
Although some believe that you don’t need to bother with link removals, I feel you are doing your client a disservice by not even attempting to get some of their bad links removed. That is a debate for another day however; assuming you do wish to do link removals, here are some budget-friendly methods.
Citation Labs Contact Finder - $10
Citation Labs have a whole range of tools, and you pay for bandwidth depending on how many requests you need to process. The tool that is particularly useful for link removal work is the Contact Finder, which returns contact form URLs as well as email addresses. You start with 10Mb of free bandwidth, which can search contact details for around 300-400 domains, but you can purchase 100Mb for only $10 – which should be sufficient for most projects.
Tout – $0 (Free Trial)
Once you have all your contact details, you’ll need to email all the site owners to ask them to remove the links. Tout is a great tool for automating this process, as it allows you to bulk upload the data and use dynamic templates to populate each email. It will also tell you who has opened and clicked your email, so you know who is ignoring you entirely. I tend to follow-up twice more after the initial request, using different subject headers, typically around 5 days apart – meaning it is possible to complete all your removal requests within the 14 day trial period.
One of the useful side-effects of this mass mailing is that many of the email addresses will fail, giving you an inbox full of delivery failure notices. In terms of collecting evidence for a reconsideration request for a manual penalty, these screenshots can serve as further evidence to Google that you have carried out plenty of work.
Tip: Make sure to keep a detailed spreadsheet of every website you contacted and the result.
Total spend so far – $15.
Do It Yourself Disavow ($0)
For some reason, one of the ‘selling’ points of the premium tools is the generation of a ‘Google friendly’ disavow list. This is probably because lots of webmasters do it wrong – but the process is really very straightforward.
You have already classified all your links, so you know which ones you don’t want Google to count. Use this as the basis to build your file, again working in Excel:
- Paste all of your URLs into a fresh worksheet, then extract the domain if you don’t already have it
- Remove duplicates by domain
- In the second column, type ‘domain:’ and fill down all the rows
- In the third column, use the CONCATENATE function to merge the two cells (see image below)
- Copy and paste all the values from column C into a Notepad document and save the file
You then need to upload your newly created disavow file for Google to process. You’ll need to be signed in to Google Webmaster Tools as the site owner and visit this page. Upload your file and submit, and the page should then look something like this:
Make sure to check the notification underneath which will tell you if you’ve made any errors during the creation of your disavow file – check that the number of domains matches the number you expected.
Total spend so far – $15.
How To Write Reconsideration Requests
Without going into too much detail, here are some brief pointers on how to write a reconsideration request, based on my experience:
- Use a bullet point list of all the work you have carried out, emphasizing headline figures (e.g. 257 links removed, 1200 site owners contacted, 3548 emails sent).
- Include as much thorough documentation as you possibly can – link to Google Drive spreadsheets of removal data and Dropbox folders of email failures and contact page screenshots.
- Don’t submit your reconsideration request too soon – if you send it one week after you got your manual penalty it won’t seem like you have worked very hard, so leave it a couple of months.
- Use polite, concise, and factual language – keep it brief and to the point.
- Don’t lie.
The main thing to remember with manual penalties (and reconsideration requests) is that your email is being read by a human. They don’t care about your history, or your story, or your ‘future plans’. They care about the fact that you have been manipulating Google’s algorithm, and want to see plenty of evidence that you’ve repented and tried to right those wrongs.
Penalty Removal for 15 Dollars
This is deliberately not the title of my post, simply because many penalties take several attempts to get removed or revoked, so I would imagine most cases would exceed this $15. Of course, this also doesn’t take into account the value of your time, the cost of which would depend primarily upon the size of the cleanup operation required.
I’ve also got a confession to make – we use paid tools all the time for our link removal work, since the more you do, the more these tools save you time and begin to pay for themselves. For the cash-strapped site owner or freelancer, however, this method will see you through.
Featured image credit: Screenshot taken 17/02/2014 from wordle.net