Google: Sites Need to Be Worthwhile to be Indexed via @sejournal, @martinibuster

Google’s John Mueller offered more information about that pesky “Discovered – but currently not indexed” message in Google Search Console. Mueller answered a series of tweets about what’s going on at Google when they choose to not index a URL.

Is Google Indexing Bugged?

There are numerous discussions on Twitter and Facebook about notices that a URL has been discovered but not indexed because it’s troubling to work on content and see it appear to be rejected by Google.

Search Marketing professional Dan Shure (@dan_shure) started a Twitter thread about this topic sharing how a new article was discovered but not indexed.

Dan shared the example of a client site that published two articles and days went by showing up as Discovered but currently not indexed.

They submitted the URLs for a re-crawl but Google essentially turned its back on those pages, refusing to index them.

Could the URL Be Blocked After Being Discovered?

So Dan floats the idea that maybe the URL itself, after being discovered and not indexed, is burned at this point and decides to remove the old URL and paste the content on a new URL.

Advertisement

Continue Reading Below

That’s a really good idea to scrap the old URL and basically try again with a different URL.

He tweeted:

“So by last Friday we’d been waiting 10+ days (even new posts were being indexed)

I thought, is the *URL itself* ‘dinged’?

So we deleted one of the posts, copied/pasted the same content exactly & re-published on a new page with a slightly different URL & new publish date “

Dan continued in the next tweet:

“The post on the new URL (but same content) indexes IMMEDIATELY (without even submitting it to GSC) in just a few hours.

The OTHER post which we left alone still was not indexed though.

We’re going to move that one to a new URL/date now and see if the same thing happens.”

Dan concluded that there must have been something about that URL.

Dan explained:

“We know Google indexes content using the URL as basically the main “id” by which all signals are associated.. so it could make sense if a URL gets “Discovered but not indexed” on the first pass it gets some sort of “ding” – maybe something to try if you run into this problem”

Nothing New or Special About Discovered/Not Indexed

Google’s John Mueller questioned whether the tool was confusing people and whether Google should just remove it.

Teetering on the Edge of Indexing

Mueller acknowledged that sometimes the same content will get indexed under a new URL in situations where the site is “teetering on the edge of indexing” which could mean many things such as overall site quality.

That’s a really useful bit of insight Mueller shared:

“Yeah, that can happen. But it can also happen that it drops out again a week later, or a different URL drops out.

If you’re teetering on the edge of indexing, there’s always fluctuation.

It basically means you need to convince Google that it’s worthwhile to index more.”

John followed up with a deeper explanation: 

“Since we don’t have an understanding of the URL (it’s not indexed), we have to pull in the rest of the site to better understand its potential context within the site, and within the rest of the web. Is it something the web has been waiting for? Or is it just another red widget?”

When pressed to explain how to convince Google to index something John Mueller responded with:

“Awesomeness.
Lots of awesomeness.
All kinds of awesomeness.
And add more awesomeness.”

What is Awesomeness?

Be awesome makes sense and doesn’t need explanation. But it is kind of vague.

I prefer something like: Don’t do what everyone else is doing, just do it how you feel is best.

Or something like: Create something that would have made you excited when you were new to the topic.

More is Not Better

That whole thing about do it “Ten Times Better” is motivational but naive because more does not equal better.

Another common mistake is to copy the competitors, to use their same keywords and synonyms (as if there’s magic in their keywords) because that leads to essentially rewritten content.

Prince didn’t become famous by copying Michael Jackson, right?

So why are SEOs so hung up on taking inspiration from what already ranks in the top ten?  It makes sense to see what Google is ranking but it stops making sense when an SEO starts to rewrite what is already ranking.

If you know the topic and are good at it, then why not try not looking at what the competitors are doing and just do your best? Maybe Google might recognize a singular voice in your site, others will call it awesome and it probably won’t have issues getting indexed.

Citation

Read the Twitter discussion:

Could “Discovered – but currently not indexed” put a URL in some sort of ‘blacklist’?

Searchenginejournal.com

Roger Montti

Roger Montti is a search marketer with 20 years experience.
I offer site audits and link building strategies.

www.searchenginejournal.com/author/roger-montti/