SEARCHENGINES
Google Favicon Crawler Uses Googlebot & Googlebot-Image Token
Google has updated its Google crawler help documentation to specific that the Google Favicon crawler now respects the Googlebot and Googlebot-Image robots rules. Previously, it seemed to just respect its own Google Favicon rules but that was changed or updated a few days ago.
The docs from earlier this year show this:
The revised document shows this:
Dave Smart, where I found this from, said on Twitter “FYI: Looks like the favicon crawler from google now respects Googlebot-Image / Googlebot robots.txt rules, (followed it own, Googlebot favicon, before apparently) so perhaps make sure you’re not blocking it accidentally now.”
Forum discussion at Twitter.
You must be logged in to post a comment Login