SEARCHENGINES

Google Deduplication On Top Stories From Web Search Results

Published

2 years ago

January 20, 2022

Google Deduplication On Top Stories From Web Search Results

We know Google has deduplication efforts for when a featured snippet is displayed so that it might remove the snippet from the main web results. Google may do the same, in some cases, for when a URL is listed in the top stories section and then not show it in the main web results.

Google in some cases may be removing the web search result snippet when it shows that same result in the top stories section of the Google search results.

Update: Here is how deduping works for top stories:

Just to cap off with the further clarification I promised, we deduplicate a link from web results if a link appears as the first link in Top Stories and if the Top Stories box appears before web results. If it comes after, we don’t. And again, it’s something we’re reviewing.

— Danny Sullivan (@dannysullivan) January 19, 2022

Rest of the story continued but the update is in the tweet above.

Danny Sullivan of Google said this on Twitter after a complaint from The Verge. Danny said “it’s showing in Top Stories, it is getting deduplicated from the rest of the page. Deduplication can often be useful. Doing this search in the way that user might by using solution-seeking terms rather than unusual terms in the headline, there you are at the top in Top Stories plus deduplicating means there’s more variety from other publications. In searches like that, our systems also are going to generally seek to show the most helpful, reliable info they can. That’s why you don’t see a lot of duplicates of your article showing. Duplicates certainly exist, but it isn’t that helpful to show them. That leads to headline-oriented searches. As I said before, that’s super common among authors. I used to do it all the time, myself. But headline searches contain typically contain a lot of terms, so our systems shift to return pages that have those terms. his means authors are more likely to find duplicates, even though for typical searches that readers would do, these are unlikely to appear. But our deduplication feature may still kick in even for these, as was happening in this case. As I said, deduplication can be helpful. But we also understand the concern this might be raising. We’ve been doing this with Top Stories since last May, but we’re going to revisit this to see if we should continue or perhaps make other changes. Also, I’m still checking, but I believe this deduplication is especially unique in that it only happens with Top Stories if there’s a single story shown or perhaps only for the very first story shown.”

So you can see, Danny is giving this as an excuse for why other publications are in the web results and not The Verge for that query. But as you can also see, he seems to explain that sometimes it does not work this way.

Deduplication can often be useful. Doing this search in the way that user might by using solution-seeking terms rather than unusual terms in the headline, there you are at the top in Top Stories plus deduplicating means there’s more variety from other publications…. pic.twitter.com/638IAZLWIV

— Danny Sullivan (@dannysullivan) January 18, 2022

The interesting part is it does seem that Google is indeed deduping here when it comes to top stories.

The funny thing, hours before this, News SEO Barry Adams did a whole Twitter thread on this:

This is not the case. It can be easily disproven; look for recent articles published on a major news outlet (BBC, The Guardian, NYT, etc). Search for relevant keywords relating to that article.

Chances are, you’ll see the article in the news box as well as on the regular SERP. pic.twitter.com/3u7HH7jppq

— Barry Adams 📰 (@badams) January 18, 2022

So a newly published article has had very little opportunity to rank in regular SERPs. It may show in Top Stories boxes, but ranking in regular SERPs could take days or weeks or even months.

Plus, authoritative publishers have an edge here, because, well, they’re authoritative.

— Barry Adams 📰 (@badams) January 18, 2022

In summary; there is no ‘news’ filter on Google SERPs. Articles can and do show up in both Top Stories and regular SERPs.

But often this doesn’t happen due to issues relating to speed, intent, and competition.

/end

— Barry Adams 📰 (@badams) January 18, 2022

So I pointed out to Barry Adams the tweet from Danny and his response:

Colour me shocked, Danny being wrong on something. 🤪

— Barry Adams 📰 (@badams) January 18, 2022

So I did some sample searches and Google does seem to remove a recent story from this site from the top stories and show it in web results, and at the same time, remove those stories in top stories from the web results. Notice SER is not in top stories here but is in the web results and at the same time SEL and SEJ is in the top stories but not in the web results (click to enlarge):

But yes, the story does show in the news tab:

But sometimes Google is not deduplicating these top stories, a query for [the hidden resignation] shows the Business Insider story in top stories in some browsers twice and some once (thanks Glenn Gabe for the query and discussing it with me):

Deduped:

Not Deduped:

Here is another example from Glenn:

Another good example of a url showing up in Top Stories that also ranks in the 10 blue links. But from what Danny explained yesterday, Google’s deduplication system for Top Stories is nuanced. So it’s not going to happen all of the time (which makes sense based on other queries). pic.twitter.com/ywiwiQnEfw

— Glenn Gabe (@glenngabe) January 19, 2022

So it is not clear when and why Google might deduplicate a URL from showing in web search when it also shows in top stories. It might be a timing issue or something else.

Danny Sullivan did say “Also, I’m still checking, but I believe this deduplication is especially unique in that it only happens with Top Stories if there’s a single story shown or perhaps only for the very first story shown.”

Also, I’m still checking, but I believe this deduplication is especially unique in that it only happens with Top Stories if there’s a single story shown or perhaps only for the very first story shown.

— Danny Sullivan (@dannysullivan) January 18, 2022

We hope to get some clarity soon.

Forum discussion at Twitter.

Source link

Related Topics:Deduplication Google Results Search stories top Web

Up Next

Recipe Schema Markup Drops Ranges For Cook, Prep & Total Time

Don't Miss

Daily Search Forum Recap: January 19, 2022

Click to comment

You must be logged in to post a comment Login

SEARCHENGINES

Generative Summaries For Search Results

Published

4 hours ago

December 1, 2023

Entireweb News Bot

Google has a published patent named “Generative summaries for search results” which is believed to be the patent behind the Search Generative Experience launch we saw earlier this year. This patent was filed on March 20, 2023 and approved on September 26, 2023 under the patent ID US11769017B1.

Juan Gonzalez Villa posted a thread on X breaking it down, which I will embed below so you can read it.

The abstract reads:

At least selectively utilizing a large language model (LLM) in generating a natural language (NL) based summary to be rendered in response to a query. In some implementations, in generating the NL based summary additional content is processed using the LLM. The additional content is in addition to query content of the query itself and, in generating the NL based summary, can be processed using the LLM and along with the query content—or even independent of the query content. Processing the additional content can, for example, mitigate occurrences of the NL based summary including inaccuracies and/or can mitigate occurrences of the NL based summary being over-specified and/or under-specified.

Here are Juan’s posts:

Before we start: there are currently no other patents assigned to Google mentioning generative AI techniques and search.

There might be other patents on the same topic still pending, but I believe this document is highly relevant to Google SGE as it works right now. pic.twitter.com/L4Ly5Ows2c

— Juan González Villa (@seostratega) November 29, 2023

1. Receiving a query associated with a client device. The query can be explicitly entered by a user or automatically generated based on context.

2. Selecting a set of search result documents (SRDs) that are responsive to the query and related or recent queries.

— Juan González Villa (@seostratega) November 29, 2023

4. Processing the SRD content snippets using an LLM to generate LLM output. An optional summarization prompt can also be included.

5. Generating a natural language summary using the LLM output. This leverages robustness of the LLM while constraining it to the SRD content.

— Juan González Villa (@seostratega) November 29, 2023

This checks out with something stated in the document “An Overview of SGE”, made public by Google around SGE’s launch, although we didn’t have any more details.

The patent now provides plenty of detail around how and why several models are available and can be used: pic.twitter.com/U035N35Bxl

— Juan González Villa (@seostratega) November 29, 2023

3. The selection can be based on:

• Processing the query with a classifier to predict best model(s)
• Detecting certain terms in the query to indicate suitable model(s)
• Considering search result types/content to determine appropriate model(s)

— Juan González Villa (@seostratega) November 29, 2023

So by dynamically selecting from multiple candidate generative models, the invention aims to optimize accuracy and efficiency by choosing the most suited model(s) for any given query.

The patent also provide some details on how the links to sources work:

— Juan González Villa (@seostratega) November 29, 2023

1. “A portion, of a visually rendered NL summary, that is supported by a first SRD can be selectable (and optionally underlined, highlighted, and/or otherwise annotated).

A selection of the portion can result in navigating to a link corresponding to the first SRD.”

— Juan González Villa (@seostratega) November 29, 2023

3. The links can be general links to the SRDs or specific anchor links to the portions that provide the verification.

4. The linkified portions can be determined based on comparing the summary content to SRD content using encoder models to identify verified portions. pic.twitter.com/zlJFFO2c6Q

— Juan González Villa (@seostratega) November 29, 2023

Here’s how confidence works, according to the patent:

1. Confidence measures can be generated for portions of the summary or for the summary as a whole.

The confidence measures are then used to determine which confidence annotation from a set of candidates should be applied.

— Juan González Villa (@seostratega) November 29, 2023

3. A textual “high confidence”, “medium confidence”, or “low confidence” annotation can be annotated for the NL based summary as a whole.

Each of the portions of the NL based summary can be annotated with a corresponding color that reflects a degree of confidence in that portion

— Juan González Villa (@seostratega) November 29, 2023

Thanks for reading so far.

Here’s the link to the patent: https://t.co/AtGMiWqzG9

Now, I’ll explain how I found out about this patent, and another interesting thing:

What Google SGE and Featured Snippets have in common… 🤓⤵

— Juan González Villa (@seostratega) November 29, 2023

According to his own career summary, he worked in SGE at Google between Nov 2022 and Sep 2023. He is now a Google Fellow, the highest rung on the ladder for Google engineers, which they reach thanks to “consistently outstanding accomplishments”. pic.twitter.com/aZmpfkajmI

— Juan González Villa (@seostratega) November 29, 2023

Their careers afterwards were very similar: they both went to Apple, where they worked in Search and Siri, and came back to Google at the end of 2022 to work in SGE.

So… how did I find the patent? I decided to look for patents by any of these two engineers and there it was. 💡 pic.twitter.com/1uUPi9myoj

— Juan González Villa (@seostratega) November 29, 2023

If you enjoyed this thread, any feedback, likes and/or RTs are appreciated.

Also, here’s a post version of this thread:https://t.co/JSt9qNcVTr

— Juan González Villa (@seostratega) November 29, 2023

Nice write up Juan!

Forum discussion at X.

Source link

SEARCHENGINES

Google Search Ranking Algorithm Volatility Today

Published

20 hours ago

November 30, 2023

Entireweb News Bot

I am seeing some signs of some big Google Search ranking volatility and shuffling today. The November core update just finished, so Google would say it is not the core update but the November reviews update is still rolling out. Or maybe this new ranking volatility is unrelated to any confirmed update – I don’t know.

Many of the tracking tools spike this morning, which means they are seeing some big ranking volatility this morning. I am also seeing some increased chatter within the SEO community but it is early, so it is limited.

Let’s start with the tools today.

Google Search Volatility Tracking Tools

Semrush:

SimilarWeb:

SERPmetrics:

Advanced Web Rankings:

Cognitive SEO (seems stalled):

Accuranker:

Mozcast (normally updates later today):

Algoroo:

SERPstat:

Wincher:

So most of the tools are showing big changes in the Google Search results.

SEO Chatter

Here is the chatter I am seeing within the SEO community from this site and WebmasterWorld:

Very slow today…

Some wild Ride traffic-wise started the last hours.
I am getting bursts of traffic for a short period of time that makes the gained
traffic while the core update is running return again.
Appears like a fight of Google core algorithm and other ongoing algorithms is going on
and from time to time, some algorithm fires and activates and takes over
and returns the traffic, while other times, another algorithm takes control and smashes the traffic back to low and renders the Google core algorithm useless.
It’s heavy volatile.

and just as the update finished, the results are shuffled again. It’s so disgusting at this point.

I feel like the “December” update has already begun…

Our UK traffic and conversions plummeted within hours of the update finishing.

Yeah we took a big downturn last week, but thought it was Google Manipulation for Black Friday.

But since the update finished, it feels like our sites are offline, especially today.

What are you all seeing?

Is this the end of the reviews update or something new?

Forum discussion at WebmasterWorld.

Source link

SEARCHENGINES

Google November 2023 Core Update Fully Rolled Out

Published

1 day ago

November 30, 2023

Entireweb News Bot

After just under 26 days, the Google November 2023 core update is finally done rolling out. It took almost two weeks longer to roll out than the average core update and rolled out during the huge shopping days on Black Friday and Cyber Monday, also overlapping the November 2023 reviews update. So it was a big deal.

Google posted the update was done at 11:32 am ET on November 28, 2023, after it started rolling out on November 2, 2023 at 3:09 PM ET.

This is the longest documented core update rollout, the previous longest core update rollout went to the August 2023 core update took 16 days, this one was 10 days longer. It wasn’t as long as most SEOs thought it would take, but it was the longest rollout of a core update. It was not the longest update in general, the December 2022 helpful content update took 38 days to roll out but it was the longest core update roll out in history.

As a reminder, the October 2023 core update started on October 5, 2023 and completed on October 19, 2023, completing two weeks prior to this November core update rolled out.

Google November 2023 Core Update Post

Here are the posts on the release times:

The roll-out of the November core update is now complete.https://t.co/geIJA2Bg8g

— Google Search Central (@googlesearchc) November 28, 2023

Documented Volatility For November 2023 Core Update

This update kicked off quickly and was super volatile early on. The chatter within the SEO community was pretty heated throughout – even during the Thanksgiving holiday break. The tools themselves seemed to calm down, even with the reviews update rolling out the following week.

There was some additional chatter in the past 24 hours about massive volatility but the tools are not picking that up and the chatter was not that insane.

The tools themselves shows volatility from November 2nd through November 17th or so. More on that later.

Google November 2023 Core Update Quick Facts:

Here are the most important things that we know right now in short form:

Name: Google November 2023 Broad Core Update

Launched: November 2, 2023 at around 3 pm ET

Rollout: Finished on November 28, 2023 at around 11:30 am ET

Targets: It looks at all types of content

Penalty: It is not a penalty, it promotes or rewards great web pages

Global: This is a global update impacting all regions, in all languages.

Impact: Google would not tell me what percentage of queries or searches were impacted by this update but so far, this seems to be a typical core update that reaches wide and the impact is fast.

Discover: Core updates impact Google Discover and other features, also feature snippets and more.

Recover: If you were hit by this, then you will need to look at your content and see if you can do better with Google’s core update advice.

Refreshes: Google will do periodic refreshes to this algorithm but may not communicate those updates in the future. Maybe this is what we saw the past couple of weeks or all those unconfirmed Google updates.

Overlapping Updates: November Core & November Reviews Updates

Unlike with the October core update, we had the October 2023 spam update roll out, where Google said if you are not spamming then you weren’t hit by the spam update, you were hit by the core update.

With a reviews update and a core update, that is a bit harder for Google to say. They are similar updates that can impact similar sites. So there was for sure some confusion between the two. It would be hard to know for sure if your site was hit by the November core update versus the November reviews update unless your site got hit in the first batch of the core update volatility before the reviews update touched down.

Google Tracking Tools On November 2023 Core Update:

Here is what the tools showed over the past month or so with this core and reviews update rollout:

Semrush:

SERPmetrics:

Advanced Web Rankings:

Accuranker:

Mozcast:

Cognitive SEO (seems stalled):

Previous Broad Core Updates

Here is a list of the most recent core updates we’ve seen since Google started to confirm them. Previously we nicknamed them Phantom updates or unconfirmed updates.