Connect with us


Why GSC Shows Crawl Errors When Page Loads Normally via @sejournal, @martinibuster



Google’s John Mueller answered a question about Google Search Console (GSC) showing crawl errors on pages that load fine in the browser. John Mueller explained that the problem is typically found on the server side and not an issue with Googlebot.

The person asking the question tried validating the web pages in Google Search Console (GSC) but the pages would stubbornly not validate. It’s like something was blocking Googlebot and serving an error message.

Here is the question:

“I had a server error on some of my pages. When I checked the few pages shown in the example they worked just fine.

I also used the validate option a few times but Google keeps marking the pages with an error.

It’s been a month since then.

I’ve waited for Googlebot to index these pages with no success.

This has affected my organic impressions and clicks.

Is there anything I can do here?”

John Mueller affirmed that if GSC shows server errors when Googlebot is crawling the web pages then they really exist and are not a bug on Google’s side.



Continue Reading Below

Here’s what John said:

“We don’t invent errors on pages. When Googlebot checks the page and there’s a server error then we really see a server error there.”

Temporary Crawl Errors

Next Mueller observed that some of these problems are temporary.

Here’s what he said:

“And it might be that this is something that is temporary on your website and if it’s temporary then with one of the future crawls we will …try that again.

And if the error is gone then we can index the page normally.”

Sometimes these errors are indeed temporary.

For example, a server might go down for maintenance, there might be issues in DNS that is taking down a part of the Internet or it can also be that the server is overloaded and preventing crawling.


But that’s not the issue that the person asking the question is experiencing.

If the web pages load normally but whenever you validate with GSC (or try to test it with one of Google’s tools like the rich results tester or mobile friendly tester) and it fails to validate, that’s an issue at the server.


Continue Reading Below

Server Issues Can Cause GSC Indexing Errors

John Mueller next suggested that the problem the person asking the question is experiencing may be server related.


“So that’s something where if you see these kinds of issues come up regularly and in particular you use the validate feature in search console and the validation comes back and says there are still server errors then that’s something I would take up with your hoster.

And try to see if there’s something that they can do to diagnose this issue to double check what might be happening here to give you a sense of how many URLs does this affect, actually.

Because sometimes it can be tricky where if Googlebot is crawling millions of pages from your website and one hundred of them have a server error then probably that’s irrelevant because like there are always some errors somewhere.


But if Googlebot is crawling 200 pages from your website and a hundred of those are server errors then that’s a little bit more concerning that something you’d probably want to fix and make sure that it doesn’t happen.

So that’s kind of the direction I would head there.

It’s not something that Google can fix.

It’s really something that you need to fix on your side with regards to the hosting.”

Diagnosing Googlebot Crawl Errors

There’s a diagnostic trick that you can do to ascertain if this is a server-wide configuration issue, if your IP is being shared by other sites on the server.

What can be done is to identify the IP address that the site is on then run that ISP through a reverse IP checker that can show what other sites are hosted on that IP address.

Then you take that list of domain names and run them through one of Google’s tools like the AMP checker or Google’s rich results checker tool.

If the tool reports an error response for one or more of the domains then that may indicate that there is a server-wide error, which


Every server has a server log and that’s a good place to start diagnosing what is causing the problem. Those server logs will show the date and exact time of when an error happened as well as the IP address of the visitor that triggered the error.


Continue Reading Below

A typical issue can be an error in how a firewall is set up which might be set too strict and blocking Google.

If you don’t have access to the server logs then a call to the web hosting customer support is in order, as John Mueller recommended.


Watch the Google Office Hours Hangout

Why Does GSC Show Server Errors on Pages that Load Fine?
Watch at the 47:05 minute mark

[embedded content]



Google December Product Reviews Update Affects More Than English Language Sites? via @sejournal, @martinibuster



Google’s Product Reviews update was announced to be rolling out to the English language. No mention was made as to if or when it would roll out to other languages. Mueller answered a question as to whether it is rolling out to other languages.

Google December 2021 Product Reviews Update

On December 1, 2021, Google announced on Twitter that a Product Review update would be rolling out that would focus on English language web pages.

The focus of the update was for improving the quality of reviews shown in Google search, specifically targeting review sites.

A Googler tweeted a description of the kinds of sites that would be targeted for demotion in the search rankings:

“Mainly relevant to sites that post articles reviewing products.

Think of sites like “best TVs under $200″.com.

Goal is to improve the quality and usefulness of reviews we show users.”



Continue Reading Below

Google also published a blog post with more guidance on the product review update that introduced two new best practices that Google’s algorithm would be looking for.

The first best practice was a requirement of evidence that a product was actually handled and reviewed.

The second best practice was to provide links to more than one place that a user could purchase the product.

The Twitter announcement stated that it was rolling out to English language websites. The blog post did not mention what languages it was rolling out to nor did the blog post specify that the product review update was limited to the English language.

Google’s Mueller Thinking About Product Reviews Update

Screenshot of Google's John Mueller trying to recall if December Product Review Update affects more than the English language

Screenshot of Google's John Mueller trying to recall if December Product Review Update affects more than the English language

Product Review Update Targets More Languages?

The person asking the question was rightly under the impression that the product review update only affected English language search results.



Continue Reading Below

But he asserted that he was seeing search volatility in the German language that appears to be related to Google’s December 2021 Product Review Update.

This is his question:

“I was seeing some movements in German search as well.

So I was wondering if there could also be an effect on websites in other languages by this product reviews update… because we had lots of movement and volatility in the last weeks.

…My question is, is it possible that the product reviews update affects other sites as well?”

John Mueller answered:

“I don’t know… like other languages?

My assumption was this was global and and across all languages.

But I don’t know what we announced in the blog post specifically.


But usually we try to push the engineering team to make a decision on that so that we can document it properly in the blog post.

I don’t know if that happened with the product reviews update. I don’t recall the complete blog post.

But it’s… from my point of view it seems like something that we could be doing in multiple languages and wouldn’t be tied to English.

And even if it were English initially, it feels like something that is relevant across the board, and we should try to find ways to roll that out to other languages over time as well.

So I’m not particularly surprised that you see changes in Germany.

But I also don’t know what we actually announced with regards to the locations and languages that are involved.”

Does Product Reviews Update Affect More Languages?

While the tweeted announcement specified that the product reviews update was limited to the English language the official blog post did not mention any such limitations.

Google’s John Mueller offered his opinion that the product reviews update is something that Google could do in multiple languages.


One must wonder if the tweet was meant to communicate that the update was rolling out first in English and subsequently to other languages.

It’s unclear if the product reviews update was rolled out globally to more languages. Hopefully Google will clarify this soon.


Google Blog Post About Product Reviews Update

Product reviews update and your site

Google’s New Product Reviews Guidelines

Write high quality product reviews

John Mueller Discusses If Product Reviews Update Is Global

Watch Mueller answer the question at the 14:00 Minute Mark

[embedded content]

Continue Reading

Subscribe To our Newsletter
We promise not to spam you. Unsubscribe at any time.
Invalid email address