Try It Free
  1. SEJ
  2.  » 
  3. News

How to Get in the Mind of Google: The Human Rater Handbook Leaked

How to Get in the Mind of Google: The Human Rater Handbook Leaked

Google has a manual for their team of quality raters. The job of these raters is to analyze websites to determine their quality and then advise Google’s algorithm engineers on best practices for revising the algorithm.

Every now and then, copies of these guidelines gets leaked. It’s not likely that Google is surreptitiously masterminding these “leaks,” even though that’s possible. It’s more likely that the community of overeager SEOs and marketers are somehow getting a hold of it, and letting it slip to a wider audience.

Recently, it was leaked again, and it brings up a great point to talk about these guidelines, and what it means for content marketing and SEO.

What’s Included in the “Quality Rater Guidelines?”

The Quality Rater Guidelines is basically a manual designed to help human raters at Google do their job. Think about the manuals you have for your job. They tell you how to do certain things.They’re not terribly exciting, and you probably wouldn’t choose them for pleasure reading.

Continue Reading Below

The human raters are people at the Googleplex, or working remotely, whose job it is to comb the Internet and perform hundreds of Google searches a day. They are trying to figure out how to improve the search algorithm so it will return the highest quality results.

If you think about it, this is really the only way that an algorithm can be effectively developed. Google’s search results aren’t a result of some arbitrary algorithmic formula. Those formula have to be derived from a cognitively qualitative source — from humans.

The information in Google’s Quality Rater Guidelines is important, because it codifies the process that the human raters engage in when they analyze the results from their queries. The information included in this manual affects search process, search results, the search algorithm, and SEO.

Here’s a screenshot of the leaked document on Scribd.

human rater handbook

Is There Any New or Earthshaking Information?

If you have time to read the full 160 pages of information in the manual, then go ahead. While it’s not a cliffhanger, it could have an impact on the way you fulfill your role as an online marketer or SEO.

Continue Reading Below

The truth is, however, the rater handbook does not unleash a tidal wave of shocking information. While there is an important level of information contained in it, it’s not really anything that we don’t already know.

Matt Cutts said in 2012 “We might be able to make those human quality rater guidelines — that we make available to people at Google — available to the larger world.”  In 2013, that happened. Sort of. Google released “How Search Works” as a public version of the quality rater guidelines. Much to the protest of the SEO community, they removed a lot of information that is contained in the actual manual.

Is this a problem? Not exactly. We have the most important information available in kernel form in How Search Works. Whatever we lack is either not relevant to our purposes or not entirely impacting to search.

What we have available between How Search Works and the leaked document is nothing more than what we’ve already known for a long time — a set of web and content best practices.

I’ll draw out some of the most notable points in my discussion below.

What are Some of the Most Important Points in the Quality Rater Guidelines?

Content Needs to be Expert Level

Google’s human raters and algorithm can distinguish between content written by an amateur and content written by a pro. If you want your content to rank well, it’s got to be professionally written. If you’re looking to win in the content marketing game, don’t mess around with low-quality writers.

Go for the best. Hire writers who have expert level writing ability and a mastery of the selected topic.

Link To and Mention Other Authoritative Sources

We’ve known for a while that co-citation and coocurence are a critical part of a successful and well-ranked website. In the quality guidelines, it’s obvious that Google determines the quality of a page based on the sources that page cites.

In other words, if you cite Wikipedia, Time, New York Times, etc., that’s considered “authoritative.” If, on the other hand, you’re quoting “experts” from Yahoo Answers, that might not be considered a quality outbound link.

Continue Reading Below

Update Your Content Regularly

Combined with the freshness algorithm, Google is constantly analyzing pages for up-to-date information. This is one reason why content marketers insist on “evergreen content” in their efforts.

Is your content current and up to date? If not, don’t expect it to rank high in the search results. The quality guidelines makes this obvious: “In order for a site to be deemed high quality it must contain updated information.”

Add More Content and Pages to Your Website

Websites with scanty information get scanty search results. It’s a quality thing. Google decides that the more information a website has, the more qualitative and thus higher-rated it is.

A website with two pages is probably not going to perform as well in the SERPs as a site with 2,000 pages. The handbook puts it like this: “A high quality rating requires a satisfying amount of website information.”

Do we know what “satisfying” looks like in terms of word count or page count? No. But we do know that “satisfying” smacks of user experience, so you should furnish as much information as you think would be helpful for your users.

Continue Reading Below

Create an Optimal User Experience

In addition to the powerful impact of content, you must still give a lot of attention to optimizing the user experience. Google’s handbook states that quality pages must have a “satisfying user experience on the page and website”.

If you have frustrating navigation, load times, or other negative experiences, then you probably need to update a thing or two. This doesn’t mean you need to have anything super fancy. Google’s word is “functional”:  “A High quality rating requires…functional page design which allows users to easily focus.”

Reel in your design, and promote your content.

Provide Contact Information

Websites need legitimacy. That legitimacy is what allows users and visitors to access your information and get in touch with you if they need to. The handbook “requires…About Us information, Contact or Customer Service information, etc.”

Make it easy for people to trust you, contact you, and see that you’re legit.


If I were to sum up the entire rater’s handbook in one sentence, it would be this: Create plenty of high-quality content that serves the user as best as possible.

Continue Reading Below

That’s a soft definition. There are no word counts, image requirements, number of outbound links, or anything else. That’s what the handbook is about. Remember, it’s for human raters, not the algorithm.

And that’s the whole deal. Focus on humans, not algorithms. That’s where search is going, and has been going for a while. Your goal is not to game the algorithm, but to serve your users.

The better you do this, the better you’ll rank.


Image Credits

Featured Image: antb via Shutterstock
Screenshot taken September 2014


Subscribe to SEJ

Get our daily newsletter from SEJ's Founder Loren Baker about the latest news in the industry!

Topic(s) of Interest*
By clicking the "SUBSCRIBE" button, I agree and accept the privacy policy of Search Engine Journal.

Neil Patel

Neil Patel is the co-founder of KISSmetrics, an analytics provider that helps companies make better business decisions. Neil also blogs ... [Read full bio]

Read the Next Article
Read the Next