SEO

AI Re-Ranking For Semantic Search

Published

2 years ago

August 11, 2022

Entireweb News Bot

Search isn’t just about matching keywords – and that’s even more true when we talk about semantic search.

Semantic search is about finding the right information for the searcher at the right time.

That goes beyond finding the right keywords and concepts and speculating how searchers will interact with the results.

Artificial intelligence (AI) re-ranking will take information about the people who come to search and tailor search results to the individual.

That might be done on a cohort level, changing results based on trends, seasonality, and popularity.

It might also be done individually, changing results based on the current searcher’s desires.

While AI re-ranking is not easy to implement in a search engine, it brings outsized value for conversions and searcher satisfaction.

Re-Ranking With Artificial Intelligence

AI-driven re-ranking can improve search results, no matter the underlying ranking algorithm a search engine uses.

That’s because good search results are more than textual relevance and business metrics like raw popularity.

Good results take into account other signals and do so on a per-query level.

To see why this is important, let’s focus on the business metric of popularity.

It’s a good general ranking signal but can fall short for specific queries. A search query of “red dress” might bring up in the first results two different dresses: “backless dress with red accents” and “summer dress in bright red.”

The backless dress might be more popular as an overall dress and product.

But in this case, specifically, it’s not what customers want.

They want a red dress, not one with red accents, and they click and buy accordingly.

Shouldn’t the search engine take that as a signal to rank the summer dress higher?

Search Analytics

As the above example shows: Understanding what searchers are doing is necessary for re-ranking.

The two most common events to track are clicks and conversions.

Generally, those are the only two events necessary and must be events coming from search.

The example above also highlights another important consideration: the events should be tied to specific queries.

That allows the search engine to learn from the interplay between the different result sets and user interactions. It propels the summer dress higher in the search results for the “red dress” query.

The same product might be less popular for other queries than its neighbors.

When looking at your different events, you’ll want to weigh them differently, too.

Clicking on a result is a sign of interest while making a purchase (or any other conversion metric) is a sign of commitment.

The ranking should reflect that.

The weighting doesn’t need to be complex.

You can go as simple as saying that conversions are worth double clicks.

You should test the right ratio for your own search.

You may also want to discount events based on the result ranking at the time the searcher saw it.

We know that a result’s position influences its clickthrough rate (CTR).

Without discounting events, you may have a situation where the top results become even more entrenched because they get more interactions, which keep them ranked higher – and repeating infinitely.

Freshness And Seasonality

A simple way to combat this self-reinforcing loop is by discounting events based on the time passed since the event.

That happens because each event that occurred in the past has an increasingly small impact on re-ranking. That is, until, at some point, it has no impact at all.

For example, you might divide the impact of each event by two, each day, for 30 days. And after 30 days, stop using the event for ranking.

A nice benefit of using freshness in the re-ranking algorithm is that it also introduces seasonality into the results.

Not only do you stop recommending videos that were extremely popular years ago but are boring to people today; you also will recommend “learn how to swim” videos in the summer, and “learn to ski” videos in the winter.

YouTube has seasonality and freshness built into its algorithm precisely for this purpose.

Using Signals To Re-rank

Now that you’ve got the signals and decaying them over time, you can apply them to the search results.

When we see “artificial intelligence,” we often think of something incredibly complex and inscrutable.

AI, though, can also be as simple as taking data over time and using it to make decisions, like we’re doing here.

One easy approach is to take a certain number of results and simply re-rank them based on a score.

For performance reasons, this number of results will generally be fairly small (10, maybe 20). Then, rank them by score.

As we discussed above, the score could be as simple as adding up the number of conversions times two, plus the number of clicks.

Adding a decay function makes for more complexity, as does discounting based on result position – but the same general principle applies.

Learning To Rank

A drawback of this re-ranking system is that you are limited to re-ranking a smaller number of results.

If you have a result that would otherwise be popular but isn’t ranking high, that result won’t get the attention it warrants.

This system also requires events on the records and the queries you want to re-rank.

It won’t work for brand new product launches or user-generated content (UGC) that often comes in and out of the search index.

Learning to rank (LTR) can address these issues.

Much like the re-ranking we’ve discussed above, LTR also works based on the idea that the records searchers interact with are better than the ones they don’t.

The previous re-ranking method works by boosting or burying results directly when tied to a specific query.

Meanwhile, LTR is much more flexible. It works by boosting or burying results based on other popular results.

LTR uses machine learning to understand which queries are similar (e.g., “video games” and “gaming console”).

It can then re-rank results on the less popular queries based on interactions on the more common ones.

LTR doesn’t only generalize on queries; it generalizes on records, too.

The LTR model learns that a certain type of result is popular; for example, the Nintendo Switch game “Legend of Zelda: Breath of the Wild.”

Then, it can start to connect to other similar results (for example, “Legend of Zelda: Skyward Sword”) and boost those.

Why, then, not just use LTR if it appears to be much more powerful than your typical re-ranking and provides more query and record coverage?

(In other words: It generalizes better.)

In short, LTR is much more complex and needs more specialized in-house machine learning (ML) expertise.

Additionally, understanding why certain results are ranked in certain places is more difficult.

With the first type of re-ranking, you could look at the number of clicks and conversions over time for one record compared to another.

Meanwhile, with LTR, you have an ML model that makes connections that may not always be obvious.

(Are “Breath of the Wild” and “Sonic Colors” really all that similar?)

Personalization

While re-ranking works across all searchers, personalization is what it sounds like: personal.

The goal of personalization is to take results that are already relevant and re-rank them based on personal tastes.

While there is a debate on how much web search engines like Google use personalization in their results, personalization often impacts the performance of results in on-site search engines.

It is a useful mechanism for increasing search interactions and conversions from search.

Search Analytics

Just as with re-ranking, personalization depends on understanding how users interact with search results.

By tracking clicks and conversions, you’ll have a clearer idea of the kinds of results that the user wants to see.

One significant difference between re-ranking and personalization on this front is that, depending on your search, you might want to adjust how you apply personalization.

For example, if you sell groceries, you definitely want to recommend previously purchased products.

But if your website sells books, you won’t want to recommend a book that a customer has already bought. Indeed, you may even want to move those books down in the search results.

It’s also true, however, that you shouldn’t push personalization so hard that users only see what they’ve interacted with before.

Search empowers both finding and discovery. So, if they return to the search bar, you should be open to the possibility that they want to see something new.

Don’t rank results exclusively via personalization; make it a mix with other ranking signals.

Just as with re-ranking, personalization also benefits from event decay.

Decreasing the impact of older events makes a search more accurately represent a user’s current tastes.

In a way, you can think of it as personal seasonality.

Personalization Across Users

The kind of personalization we’ve seen so far is based on an individual’s own interactions, but you can also combine it with what others are doing inside search.

This approach shows an outsized impact on situations where the user hasn’t interacted with the items in the search results before.

Because the user doesn’t interact with the search result items, you can’t boost or bury based on past interactions, by definition.

Instead, you can look at users that are similar to the current user and then personalize based on what they have interacted with.

For example, say you have a user who has never come to you for dresses but has purchased many handbags.

Then, you can look for other users who have similar tastes and have also interacted with dresses.

Intuitively, other customers who like the same type of handbags as our searcher should also like the same dresses.

Re-Ranking And Personalization For Discovery

Search is only one example of where re-ranking and personalization can make an impact. You can use these same tools for discovery as well.

The secret is to think of your home page and category pages as search results.

Then, it’s clear that you can use the same tools you use for search and gain the same benefits.

For example, a home page is similar to a search page without a query, isn’t it? And a category landing page sure does look like a search page with a category filter applied to it.

If you add personalization and re-ranking to these pages, they can be less static. They will serve users what they prefer to see, and they can push items higher that are more popular with customers overall.

And don’t worry, personalization and re-ranking can mix with editorial decisions on these pages or inside search.

The best way to handle this is by fixing the desired results in certain places and re-rank around them.

We’ve seen that personalization and re-ranking are two approaches that take user interactions with relevant signals to make search better.

You can let your user base influence the result by using the interactions.

Little by little, these interactions tell the search engine what items should be ranking higher.

Ultimately, searchers benefit from a better search experience, and you benefit from more clicks and conversions.

More resources:

Featured Image: amasterphotographer/Shutterstock

Source link

Related Topics:ReRanking Search Semantic seo

Up Next

Nine features your clients will dig

Don't Miss

How do videos help your SEO strategy?

Click to comment

You must be logged in to post a comment Login

SEO

WordPress Insiders Discuss WordPress Stagnation

Published

4 hours ago

August 31, 2024

Max

WordPress Insiders Discuss WordPress Stagnation

A recent webinar featuring WordPress executives from Automattic and Elementor, along with developers and Joost de Valk, discussed the stagnation in WordPress growth, exploring the causes and potential solutions.

Stagnation Was The Webinar Topic

The webinar, “Is WordPress’ Market share Declining? And What Should Product Businesses Do About it?” was a frank discussion about what can be done to increase the market share of new users that are choosing a web publishing platform.

Yet something that came up is that there are some areas that WordPress is doing exceptionally well so it’s not all doom and gloom. As will be seen later on, the fact that the WordPress core isn’t progressing in terms of specific technological adoption isn’t necessarily a sign that WordPress is falling behind, it’s actually a feature.

Yet there is a stagnation as mentioned at the 17:07 minute mark:

“…Basically you’re saying it’s not necessarily declining, but it’s not increasing and the energy is lagging. “

The response to the above statement acknowledged that while there are areas of growth like in the education and government sectors, the rest was “up for grabs.”

Joost de Valk spoke directly and unambiguously acknowledged the stagnation at the 18:09 minute mark:

“I agree with Noel. I think it’s stagnant.”

That said, Joost also saw opportunities with ecommerce, with the performance of WooCommerce. WooCommerce, by the way, outperformed WordPress as a whole with a 6.80% year over year growth rate, so there’s a good reason that Joost was optimistic of the ecommerce sector.

A general sense that WordPress was entering a stall however was not in dispute, as shown in remarks at the 31:45 minute mark:

“… the WordPress product market share is not decreasing, but it is stagnating…”

Facing Reality Is Productive

Humans have two ways to deal with a problem:

Acknowledge the problem and seek solutions
Pretend it’s not there and proceed as if everything is okay

WordPress is a publishing platform that’s loved around the world and has literally created countless jobs, careers, powered online commerce as well as helped establish new industries in developing applications that extend WordPress.

Many people have a stake in WordPress’ continued survival so any talk about WordPress entering a stall and descent phase like an airplane that reached the maximum altitude is frightening and some people would prefer to shout it down to make it go away.

Acknowledging facts and not brushing them aside is what this webinar achieved as a step toward identifying solutions. Everyone in the discussion has a stake in the continued growth of WordPress and their goal was to put it out there for the community to also get involved.

The live webinar featured:

Miriam Schwab, Elementor’s Head of WP Relations
Rich Tabor, Automattic Product Manager
Joost de Valk, founder of Yoast SEO
Co-hosts Matt Cromwell and Amber Hinds, both members of the WordPress developer community moderated the discussion.

WordPress Market Share Stagnation

The webinar acknowledged that WordPress market share, the percentage of websites online that use WordPress, was stagnating. Stagnation is a state at which something is neither moving forward nor backwards, it is simply stuck at an in between point. And that’s what was openly acknowledged and the main point of the discussion was understanding the reasons why and what could be done about it.

Statistics gathered by the HTTPArchive and published on Joost de Valk’s blog show that WordPress experienced a year over year growth of 1.85%, having spent the year growing and contracting its market share. For example, over the latest month over month period the market share dropped by -0.28%.

Crowing about the WordPress 1.85% growth rate as evidence that everything is fine is to ignore that a large percentage of new businesses and websites coming online are increasingly going to other platforms, with year over year growth rates of other platforms outpacing the rate of growth of WordPress.

Out of the top 10 Content Management Systems, only six experienced year over year (YoY) growth.

CMS YoY Growth

Webflow: 25.00%
Shopify: 15.61%
Wix: 10.71%
Squarespace: 9.04%
Duda: 8.89%
WordPress: 1.85%

Why Stagnation Is A Problem

An important point made in the webinar is that stagnation can have a negative trickle-down effect on the business ecosystem by reducing growth opportunities and customer acquisition. If fewer of the new businesses coming online are opting in for WordPress are clients that will never come looking for a theme, plugin, development or SEO service.

It was noted at the 4:18 minute mark by Joost de Valk:

“…when you’re investing and when you’re building a product in the WordPress space, the market share or whether WordPress is growing or not has a deep impact on how easy it is to well to get people to, to buy the software that you want to sell them.”

Perception Of Innovation

One of the potential reasons for the struggle to achieve significant growth is the perception of a lack of innovation, pointed out at the 16:51 minute mark that there’s still no integration with popular technologies like Next JS, an open-source web development platform that is optimized for fast rollout of scalable and search-friendly websites.

It was observed at the 16:51 minute mark:

“…and still today we have no integration with next JS or anything like that…”

Someone else agreed but also expressed at the 41:52 minute mark, that the lack of innovation in the WordPress core can also be seen as a deliberate effort to make WordPress extensible so that if users find a gap a developer can step in and make a plugin to make WordPress be whatever users and developers want it to be.

“It’s not trying to be everything for everyone because it’s extensible. So if WordPress has a… let’s say a weakness for a particular segment or could be doing better in some way. Then you can come along and develop a plug in for it and that is one of the beautiful things about WordPress.”

Is Improved Marketing A Solution

One of the things that was identified as an area of improvement is marketing. They didn’t say it would solve all problems. It was simply noted that competitors are actively advertising and promoting but WordPress is by comparison not really proactively there. I think to extend that idea, which wasn’t expressed in the webinar, is to consider that if WordPress isn’t out there putting out a positive marketing message then the only thing consumers might be exposed to is the daily news of another vulnerability.

Someone commented in the 16:21 minute mark:

“I’m missing the excitement of WordPress and I’m not feeling that in the market. …I think a lot of that is around the product marketing and how we repackage WordPress for certain verticals because this one-size-fits-all means that in every single vertical we’re being displaced by campaigns that have paid or, you know, have received a a certain amount of funding and can go after us, right?”

This idea of marketing being a shortcoming of WordPress was raised earlier in the webinar at the 18:27 minute mark where it was acknowledged that growth was in some respects driven by the WordPress ecosystem with associated products like Elementor driving the growth in adoption of WordPress by new businesses.

They said:

“…the only logical conclusion is that the fact that marketing of WordPress itself is has actually always been a pain point, is now starting to actually hurt us.”

Future Of WordPress

This webinar is important because it features the voices of people who are actively involved at every level of WordPress, from development, marketing, accessibility, WordPress security, to plugin development. These are insiders with a deep interest in the continued evolution of WordPress as a viable platform for getting online.

The fact that they’re talking about the stagnation of WordPress should be of concern to everybody and that they are talking about solutions shows that the WordPress community is not in denial but is directly confronting situations, which is how a thriving ecosystem should be responding.

Watch the webinar:

Is WordPress’ Market share Declining? And What Should Product Businesses Do About it?

Featured Image by Shutterstock/Krakenimages.com

Source link

SEO

Google’s New Support For AVIF Images May Boost SEO

Published

23 hours ago

August 31, 2024

Max

Google's New Support For AVIF Images May Boost SEO

Google announced that images in the AVIF file format will now be eligible to be shown in Google Search and Google Images, including all platforms that surface Google Search data. AVIF will dramatically lower image sizes and improve Core Web Vitals scores, particularly Largest Contentful Paint.

How AVIF Can Improve SEO

Getting pages crawled and indexed are the first step of effective SEO. Anything that lowers file size and speeds up web page rendering will help search crawlers get to the content faster and improve the amount of pages crawled.

Google’s crawl budget documentation recommends increasing the speeds of page loading and rendering as a way to avoid receiving “Hostload exceeded” warnings.

It also says that faster loading times enables Googlebot to crawl more pages:

Improve your site’s crawl efficiency

Increase your page loading speed
Google’s crawling is limited by bandwidth, time, and availability of Googlebot instances. If your server responds to requests quicker, we might be able to crawl more pages on your site.

What Is AVIF?

AVIF (AVI Image File Format) is a next generation open source image file format that combines the best of JPEG, PNG, and GIF image file formats but in a more compressed format for smaller image files (by 50% for JPEG format).

AVIF supports transparency like PNG and photographic images like JPEG does but does but with a higher level of dynamic range, deeper blacks, and better compression (meaning smaller file sizes). AVIF even supports animation like GIF does.

AVIF Versus WebP

AVIF is generally a better file format than WebP in terms of smaller files size (compression) and image quality. WebP is better for lossless images, where maintaining high quality regardless of file size is more important. But for everyday web usage, AVIF is the better choice.

Is AVIF Supported?

AVIF is currently supported by Chrome, Edge, Firefox, Opera, and Safari browsers. Not all content management systems support AVIF. However, both WordPress and Joomla support AVIF. In terms of CDN, Cloudflare also already supports AVIF.

I couldn’t at this time ascertain whether Bing supports AVIF files and will update this article once I find out.

Current website usage of AVIF stands at 0.2% but now that it’s available to surfaced in Google Search, expect that percentage to grow. AVIF images will probably become a standard image format because of its high compression will help sites perform far better than they currently do with JPEG and PNG formats.

Research conducted in July 2024 by Joost de Valk (founder of Yoast, ) discovered that social media platforms don’t all support AVIF files. He found that LinkedIn, Mastodon, Slack, and Twitter/X do not currently support AVIF but that Facebook, Pinterest, Threads and WhatsApp do support it.

AVIF Images Are Automatically Indexable By Google

According to Google’s announcement there is nothing special that needs to be done to make AVIF image files indexable.

“Over the recent years, AVIF has become one of the most commonly used image formats on the web. We’re happy to announce that AVIF is now a supported file type in Google Search, for Google Images as well as any place that uses images in Google Search. You don’t need to do anything special to have your AVIF files indexed by Google.”

Read Google’s announcement:

Supporting AVIF in Google Search

Featured Image by Shutterstock/Cast Of Thousands

Source link

SEO

CMOs Called Out For Reliance On AI Content For SEO

Published

2 days ago

August 30, 2024

Max

CMOs Called Out For Reliance On AI Content For SEO

Eli Schwartz, Author of Product-Led SEO, started a discussion on LinkedIn about there being too many CMOs (Chief Marketing Officers) who believe that AI written content is an SEO strategy. He predicted that there will be reckoning on the way after their strategies end in failure.

This is what Eli had to say:

“Too many CMOs think that AI-written content is an SEO strategy that will replace actual SEO.

This mistake is going to lead to an explosion in demand for SEO strategists to help them fix their traffic when they find out they might have been wrong.”

Everyone in the discussion, which received 54 comments, strongly agreed with Eli, except for one guy.

What Is Google’s Policy On AI Generated Content?

Google’s policy hasn’t changed although they did update their guidance and spam policies on March 5, 2024 at the same time as the rollout of the March 2024 Core Algorithm Update. Many publishers who used AI to create content subsequently reported losing rankings.

Yet it’s not said that using AI is enough to merit poor rankings, it’s content that is created for ranking purposes.

Google wrote these guidelines specifically for autogenerated content, including AI generated content (Wayback machine copy dated March 6, 2024)

“Our long-standing spam policy has been that use of automation, including generative AI, is spam if the primary purpose is manipulating ranking in Search results. The updated policy is in the same spirit of our previous policy and based on the same principle. It’s been expanded to account for more sophisticated scaled content creation methods where it isn’t always clear whether low quality content was created purely through automation.

Our new policy is meant to help people focus more clearly on the idea that producing content at scale is abusive if done for the purpose of manipulating search rankings and that this applies whether automation or humans are involved.”

Many in Eli’s discussion were in agreement that reliance on AI by some organizations may come to haunt them, except for that one guy in the discussion

Read the discussion on LinkedIn:

Too many CMOs think that AI-written content is an SEO strategy that will replace actual SEO

Featured Image by Shutterstock/Cast Of Thousands

Source link