Blocking USA Users But Allowing Googlebot Is Against Google’s Webmaster Guidelines

Google sends its crew of crawlers, Googlebot, from the United States. That is where 99% of the crawling originates from. If you are not allowed to show your website to users based in the United States, then you also cannot show that website to Googlebot.

You cannot show content to Googlebot when a user trying to access that same page will not see that content from the same region. Google’s John Mueller said it would be against Google’s webmaster guidelines to show Googlebot the content but then show a US based user a screen that says they are not allowed to see this content because of their location.

You can do the opposite, show US based users content but hide it from maybe folks in Russia with a disclaimer of sorts, and that would not be against Google’s guidelines. Why? Because Googlebot crawls from the US, not from Russia.

This is not new, we covered it numerous times but it came up again recently. Here is the recent context:

P.S. never use location-based redirects. Google mostly crawls from America and you are shooting yourself in the foot.

— Billie 🦕 (@BillieGeena) March 1, 2022

Advertisement

That would be against the webmaster guidelines.

— 🐐 John 🐐 (@JohnMu) March 1, 2022

The usual recommendation is to publish something that can be accessed from the US, and to refer to the detailed content from there. It’s not great, blocking users isn’t great, but laws/policies are hard sometimes.

— 🐐 John 🐐 (@JohnMu) March 1, 2022

So if you want to block countries outside of the US, you can do that, if you want – just saying. But you cannot block users from the US.

Forum discussion at Twitter.