Connect with us

SEARCHENGINES

Google Favicon Crawler Uses Googlebot & Googlebot-Image Token

Published

on

Google Favicon Crawler Uses Googlebot & Googlebot-Image Token


Google has updated its Google crawler help documentation to specific that the Google Favicon crawler now respects the Googlebot and Googlebot-Image robots rules. Previously, it seemed to just respect its own Google Favicon rules but that was changed or updated a few days ago.

The docs from earlier this year show this:

click for full size

The revised document shows this:

click for full size

Dave Smart, where I found this from, said on Twitter “FYI: Looks like the favicon crawler from google now respects Googlebot-Image / Googlebot robots.txt rules, (followed it own, Googlebot favicon, before apparently) so perhaps make sure you’re not blocking it accidentally now.”

Forum discussion at Twitter.





Source link

Advertisement
See also  Twitter explored buying India’s ShareChat and turning Moj into a global TikTok rival

SEARCHENGINES

Google Updates Crawl Stats Report Help Documentation

Published

on

Google Updates Crawl Stats Report Help Documentation

Google has also updated the crawl stats report help documentation last week. Google made a lot of support documentation changes in the past week including product rich results, Googlebot doc, job posting help doc, moving the HTTP status codes doc and also this crawl stats report help document.

You can see this crawl stats help document here and the changes were summed up very nicely by Brodie Clark on Twitter who said:

  • Note about robots.txt 404ing (can crawl any URL)
  • Using a 12hr – 30 day timeframe for fetching
  • Details about homepage usage

He posted the document before and after changes as well:

click for full size

Brodie wrote, “Heads-up: Google just added some interesting clarifications to their Crawl Stats Report doc.” Nice catch Brodie!

Forum discussion at Twitter.



Source: www.seroundtable.com

See also  Twitter explored buying India’s ShareChat and turning Moj into a global TikTok rival
Continue Reading

DON'T MISS ANY IMPORTANT NEWS!
Subscribe To our Newsletter
We promise not to spam you. Unsubscribe at any time.
Invalid email address

Trending

en_USEnglish