SEARCHENGINES

Google Crawl Spikes After URL Parameter Tool Went Offline?

Published

7 months ago

May 4, 2022

Google Crawl Spikes After URL Parameter Tool Went Offline?

As you know, last week, on April 26th, Google disabled the URL parameter tool within Google Search Console and stopped respecting any of the rules added to that tool. Some are asking if the recent crawl spikes they saw with Googlebot was related to this change.

Dave Smart posted on Twitter charts showing his crawl activity spiking around this time. Of course, this can easily be unrelated to the URL parameter tool. It is almost impossible to isolate this crawl spike to the URL parameter tool. But Dave did say there were no other changes around this time.

Here are his tweets:

1/3 Gone, but not forgotten! Now might be a good time to check in to see what’s being crawled.
I suspect they depreciated using what was configured here a few days back, here’s the crawl stats for a client’s sitehttps://t.co/FbBvbCWjct pic.twitter.com/yAPhONATGU

— Dave Smart (@davewsmart) April 26, 2022

3/3 Judicious use of robots.txt quickly brought this back down from 9 hits per second from gbot to 1.5 / 2 per second, & (a lot more of) those actually valuable URLs. A temporary fix, the site’s re-platforming soon anyway. If your site relied on the param tools, keep an eye open.

— Dave Smart (@davewsmart) April 26, 2022

Glenn Gabe jumped in to say it is likely unrelated:

Always a strong possibility they picked up and started crawling randomly. I agree! This started ramping up on the 13th, so nearly 2 weeks ago, no other real changes, but then nothing is ever static.

— Dave Smart (@davewsmart) April 26, 2022

And John Mueller from Google agreed:

That would be my take too. The systems automatically adjust over time too, so I wouldn’t ever expect this to be completely static.

— 🦙 johnmu.xml (personal) 🦙 (@JohnMu) April 26, 2022

Of course, anything is possible. I have heard from some others privately that they saw significant crawl spikes after the URL parameter tool went away – and yes, they used that tool heavily. But again, it is impossible to confirm without having access to Google’s end.

Have you noticed this as well? What do you all think?

Forum discussion at Twitter.

Source: www.seroundtable.com

Related Topics:Crawl Google Offline Parameter Spikes Tool URL

Up Next

Microsoft Advertising RSAs Deadline Extended, Auto-Generated Remarketing Lists & More

Don't Miss

More Google Search UI Tests With Full Width Videos & Grid Formatted Results

SEARCHENGINES

Daily Search Forum Recap: December 9, 2022

Published

4 hours ago

December 10, 2022

Entireweb News Bot

Here is a recap of what happened in the search forums today, through the eyes of the Search Engine Roundtable and other search forums on the web.

Google is testing the new topic refinements on desktop search. Google’s John Mueller dislikes HTML sitemaps a lot. Google said if you use a CDN or cloud hosting then set up tests to make sure Googlebot is not being blocked. Google says only update the date of your stories if you make significant changes, this is my 2023 version of this story. Google posted its top searches for 2022, the year in search. And I posted my weekly SEO video recap.

Search Engine Roundtable Stories:

Search News Buzz Video Recap: Google Helpful Content System Update, Continuous Scroll On Desktop & Topic Refinements

This week we had another confirmed update to the Google helpful content system; it is now global. Google also had a lot of volatility over the previous week, unrelated to this confirmed update. Google officially rolled out continuous scroll on desktop…

Google Tests +Topic Search Bar Refinements On Desktop After Launching On Mobile

Earlier this week, Google announced it has begun to roll out the new topics + sign refinement options in the English/US on mobile search results. But it seems Google is also testing them. They are not fully live on the desktop search results.

2023 Edition: Google Says Only Update The Date When Significantly Updating The Content

It is that time of year when content creators take their content that has been performing well and updates it from the 2022 edition to the 2023 edition. But updating the references of 2022 to 2023 is not enough, and Google says only do that when you make a significant update to that content.

Google: Using A CDN Or Cloud Hosting, Set Up Routine Tests To Ensure Googlebot Is Not Blocked

Gary Illyes from Google loves posting his PSAs on Mastodon. His next PSA is that if you are using a CDN or cloud hosting service, then make sure to test up a routine test to ensure that the service is not blocking Googlebot or other search engine crawlers that you want to crawl and index your site.

Google’s John Mueller Says HTML Sitemaps Should Never Be Needed

John Mueller of Google has always downplayed the importance of HTML sitemaps over the years, he is a fan of XML sitemaps, but HTML sitemaps, not so much. On Mastodon the other day he really showed his dissatisfaction of HTML sitemaps by saying that HTML sitemaps “should never be needed.”

Google 2022 Year In Search – Top Searches

It is that time of the year again when Google publishes its top searches for the year. Not going to lie, it is getting tiring, but Google does put a lot of effort into making it inspirational with the embedded video.

Arches At The Google Carpenter Chicago Office

Here is a photo from the Google Carpenter office, the office in Chicago. You can see, Google has a lot of arches throughout that office. Here are more photos of this office from Instagram embedded bel

Other Great Search Threads:

Search Engine Land Stories:

Feedback:

Have feedback on this daily recap; let me know on Twitter @rustybrick or @seroundtable, you can follow us on Facebook and make sure to subscribe to the YouTube channel, Apple Podcasts, Spotify, Google Podcasts or just contact us the old fashion way.

Source: www.seroundtable.com