Google On Making Mastodon More Search Engine Friendly

There is an interesting thread on Mastodon where Danny Sullivan and John Mueller, both of who work at Google, talk about making Mastodon more search-engine friendly. The thread was started by Gabe Rivera of Techmeme, when he wanted a way to search Mastodon.

Here is a link to part of the thread on Mastodon, where Danny Sullivan explains that Google can pick up anything on the web that is not blocked. He said, “will pick up anything on the web including Mastodon pages that aren’t blocked to search engines.” You can of course block your Mastodon posts from being indexed, Danny said “and any individual who doesn’t want their posts indexed can use Preferences > Opt-out of search engine indexing.”

Google will learn how to detect good content on this social network. Danny added, “Over time, if our systems learn there’s a lot of helpful, fresh content from a source, they tend to speed up the indexing of it. But there can be things site owners (or instance owners!) can do.”

John Mueller of Google added, “I haven’t looked into the details, but content from these servers does get indexed in Google.” “AFAIK there are RSS feeds for people & hashtags, which make discovery of public content fairly straightforward,” he added.

John then said the JavaScript-based public user interface is indexable by Google. “I’m sure someone with SEO knowledge could help improve that significantly too (it’s open-source, you don’t have that chance with closed platforms),” he added.

Since Mastodon is on multiple, many many, servers, that can be a challenge with all the different domains hosting the content. John said, “One difficulty is definitely that content is hosted on a variety of servers, individually they may choose to lock things down more. Also, since server speed affects crawl rate / crawl budget, faster servers could have it easier to get fresh content indexed.”

But “The underlying protocol (ActivityPub) is open-source, and there are lots of APIs. This makes it possible to integrate with websites (from here to there, and back), you can integrate it into your internal tools, you can create your own tracking of trending topics or sentiment analysis, etc. The problem with “you can” is also that a lot of this is not turn-key ready (and the AGPL license can make it harder for commercial projects),” he added.

Now, maybe the best way to get this content discovered by Google is to write about it on third-party sites (like this) and link to the content.

Here is a screenshot of this thread:

Forum discussion at Mastodon.