Advertisement
  1. SEJ
  2.  ⋅ 
  3. SEO

Google: If Blocking a Country’s Traffic, Don’t Block Googlebot

Site owners need to be careful not to block Googlebot if they're blocking entire countries from access their site.

Google’s John Mueller advises site owners to be careful not to block Googlebot if they choose to block traffic from an entire country.

Mueller addressed this topic twice over the past week. It came up once during the Google Search Central SEO hangout on April 9, and also came up in a Reddit thread.

During the Search Central hangout, a site owner submitted a question asking Mueller about blocking a country where they’re receiving an abundance of bot traffic.

Here is Mueller’s advice.

Google’s John Mueller on Blocking Traffic From An Entire Country

Site owners can and do block individual countries when they’re receiving problematic traffic.

Living in Switzerland, Mueller isn’t personally fond of that idea, but he understands why people do it:

“I’ve heard some people block individual countries because they see a lot of problematic traffic from individual countries. Personally I don’t really like that approach because Switzerland is one of those countries as well that people tend to say, “oh this is another European country, I will block it.”

So I don’t like it from a personal point of view. But if you’re seeing significant issues with traffic from an individual country then maybe that makes sense.”

From an SEO perspective, the key consideration here is to not block Googlebot from crawling a website.

Googlebot usually crawls sites from the US, so if a website is blocking traffic from the US it’s most likely going to cause problems.

Site owners can do a reverse DNS lookup to determine where Googlebot is crawling their site from. Mueller says it’s rare that Googlebot would crawl a site from multiple locations.

“From a search point of view we primarily crawl from the US. We don’t always crawl from the US, but we primarily do so. And essentially that means, if you need to block individual users and individual countries, then you need to make sure that you’re not blocking Googlebot, so you’re not blocking users in the US for example. So that’s kind of the search specific aspect I would watch out for there.

If you need to block individual countries, just make sure that’s not a country where Googlebot is crawling your site from. You can confirm Googlebot traffic with a reverse DNS lookup if you want to do that. But, essentially, for most websites we would crawl from the US. I think it’s very rare that we would crawl sites from other locations, and we generally don’t crawl sites from multiple locations.

So it’s not going to be the case that we would crawl from the US and from, I don’t know, Singapore or some random other country. We would generally pick one country to crawl this website from.”

Ultimately it’s up to site owners to decide if they want to block traffic from a whole country. Sites may solve their issues with problematic traffic, but they could be blocking legitimate users at the same time.

“So based on that, the important point for search is really that you’re not blocking users from the country where Googlebot is crawling from. If you’re blocking them from other countries that’s kind of up to you. It also means that you might be blocking legitimate users from those countries so I wouldn’t just randomly do that, but if you’re seeing significant issues that might be an approach.”

Hear Mueller’s full response in the video below:

Another Take on Blocking a Country’s Traffic

In the SEO space there’s an interesting phenomenon where a topic may not come up for months, and then it comes up multiple times in one week. This is one such topic.

On Reddit a site owner posted a thread asking if they can reduce their rankings in a specific country, and/or block traffic from a specific country.

The site owner notes that they receive a lot of hacking attempts from a specific country and they want to put an end to that problem.

Mueller’s advice is consistent with the advice provided during the Search Central hangout.

“You can’t reduce your rankings in one country in Google. You could set geotargeting to a different country, but that’s not certain to reduce the rankings in Indonesia (and if it did, it would affect all other countries). The only thing I’m aware of which is kinda country-limiting is video structured data, which would affect how a page is shown in video search & the video universal results (assuming you have a video on the page).

Blocking all users in a specific country is doable provided you don’t block the country where Google is crawling your site from (I’m pretty certain we don’t crawl from Indonesia). It always feels like a bad move though – you might be blocking some unwanted users, but you’re also blocking people who might be relevant to your site. IMO if you’re worried about hackers from one country, you should fix the security issues, not try to block that country. It’s trivial for anyone that’s dedicated to circumvent a country-block.”

Source: Reddit

Category News SEO
ADVERTISEMENT
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Matt G. Southern, Senior News Writer, has been with Search Engine Journal since 2013. With a bachelor’s degree in communications, ...

Google: If Blocking a Country’s Traffic, Don’t Block Googlebot

Subscribe To Our Newsletter.

Conquer your day with daily search marketing news.