SEO
How It Works & Who It’s For
For simple user queries, a search engine can reliably find the correct content using keyword matching alone.
A “red toaster” query pulls up all of the products with “toaster” in the title or description, and red in the color attribute.
Add synonyms like maroon for red, and you can match even more toasters.
But things start to become more difficult quickly: You have to add these synonyms yourself, and your search will also bring up toaster ovens.
This is where semantic search comes in.
Semantic search attempts to apply user intent and the meaning (or semantics) of words and phrases to find the right content.
It goes beyond keyword matching by using information that might not be present immediately in the text (the keywords themselves) but is closely tied to what the searcher wants.
For example, finding a sweater with the query “sweater” or even “sweeter” is no problem for keyword search, while the queries “warm clothing” or “how can I keep my body warm in the winter?” are better served by semantic search.
As you can imagine, attempting to go beyond the surface-level information embedded in the text is a complex endeavor.
It has been attempted by many and incorporates a lot of different components.
Additionally, as with anything that shows great promise, semantic search is a term that is sometimes used for search that doesn’t truly live up to the name.
To understand whether semantic search is applicable to your business and how you can best take advantage, it helps to understand how it works, and the components that comprise semantic search.
What Are The Elements Of Semantic Search?
Semantic search applies user intent, context, and conceptual meanings to match a user query to the corresponding content.
It uses vector search and machine learning to return results that aim to match a user’s query, even when there are no word matches.
These components work together to retrieve and rank results based on meaning.
One of the most fundamental pieces is that of context.
Context
The context in which a search happens is important for understanding what a searcher is trying to find.
Context can be as simple as the locale (an American searching for “football” wants something different compared to a Brit searching the same thing) or much more complex.
An intelligent search engine will use the context on both a personal level and a group level.
The personal level influencing of results is called, appropriately enough, personalization.
Personalization will use that individual searcher’s affinities, previous searches, and previous interactions to return the content that is best suited to the current query.
It is applicable to all kinds of searching, but semantic search can go even further.
On a group level, a search engine can re-rank results using information about how all searchers interact with search results, such as which results are clicked on most often, or even seasonality of when certain results are more popular than others.
Again, this displays how semantic search can bring in intelligence to search, in this case, intelligence via user behavior.
Semantic search can also leverage the context within the text.
We’ve already discussed that synonyms are useful in all kinds of search, and can improve keyword search by expanding the matches for queries to related content.
But we know as well that synonyms are not universal – sometimes two words are equivalent in one context, and not in another.
When someone searches for “football players”, what are the right results?
The answer will be different in Kent, Ohio than in Kent, United Kingdom.
A query like “tampa bay football players”, however, probably doesn’t need to know where the searcher is located.
Adding a blanket synonym that made football and soccer equivalent would have led to a poor experience when that searcher saw the Tampa Bay Rowdies soccer club next to Ron Gronkowski.
(Of course, if we know that the searcher would have preferred to see the Tampa Bay Rowdies, the search engine can take that into account!)
This is an example of query understanding via semantic search.
User Intent
The ultimate goal of any search engine is to help the user be successful in completing a task.
That task might be to read news articles, buy clothing, or find a document.
The search engine needs to figure out what the user wants to do, or what the user intent is.
We can see this when searching on an ecommerce website.
As the user types the query “jordans”, the search automatically filters on the category, “Shoes.”
This anticipates that the user intent is to find shoes, and not jordan almonds (which would be in the “Food & Snacks” category).
By getting ahead of the user intent, the search engine can return the most relevant results, and not distract the user with items that match textually, but not relevantly.
This can be all the more relevant when applying a sort on top of the search, like price from lowest to highest.
This is an example of query categorization.
Categorizing the query and limiting the results set will ensure that only relevant results appear.
Difference Between Keyword And Semantic Search
We have already seen ways in which semantic search is intelligent, but it’s worth looking more at how it is different from keyword search.
While keyword search engines also bring in natural language processing to improve this word-to-word matching – through methods such as using synonyms, removing stop words, ignoring plurals – that processing still relies on matching words to words.
But semantic search can return results where there is no matching text, but anyone with knowledge of the domain can see that there are plainly good matches.
This ties into the big difference between keyword search and semantic search, which is how matching between query and records occurs.
To simplify things some, keyword search occurs by matching on text.
“Soap” will always match “soap” or “soapy ”, because of the overlap in textual quality.
More specifically, there are enough matching letters (or characters) to tell the engine that a user searching for one will want the other.
That same matching will also tell the engine that the query soap is a more likely match for the word “soup” than the word “detergent.”
That is unless the owner of the search engine has told the engine ahead of time that soap and detergent are equivalents, in which case the search engine will “pretend” that detergent is actually soap when it is determining similarity.
Keyword-based search engines can also use tools like synonyms, alternatives, or query word removal – all types of query expansion and relaxation – to help with this information retrieval task.
NLP and NLU tools like typo tolerance, tokenization, and normalization also work to improve retrieval.
While these all help to provide improved results, they can fall short with more intelligent matching, and matching on concepts.
Semantic Search Matches On Concepts
Because semantic search is matching on concepts, the search engine can no longer determine whether records are relevant based on how many characters two words share.
Again, think about “soap” versus “soup” versus “detergent.”
Or more complex queries, like “laundry cleaner”, “remove stains clothing”, or “how do I get grass stains out of denim?”
You can even include things like image searching!
A real-world analogy of this would be a customer asking an employee where a “toilet unclogged” is located.
An employee with only a pure keyword-esque understanding of the request would fail it unless the store explicitly refers to their plungers, drain cleaners, and toilet augers as “toilet uncloggers.”
But, we would hope, the employee is wise enough to make the connection between the various terms and direct the customer to the right aisle.
(Perhaps the employee knows the different terms, or synonyms, a customer can use for any given product).
A succinct way of summarizing what semantic search does is to say that semantic search brings increased intelligence to match on concepts more than words, through the use of vector search.
With this intelligence, semantic search can perform in a more human-like manner, like a searcher finding dresses and suits when searching fancy, with not a jean in sight.
What Is Semantic Search Not?
By now, semantic search should be clear as a powerful method for improving search quality.
As such, you should not be surprised to learn that the meaning of semantic search has been applied more and more broadly.
Often, these search experiences don’t always warrant the name.
And while there is no official definition of semantic search, we can say that it is search that goes beyond traditional keyword-based search.
It does this by incorporating real-world knowledge to derive user intent based on the meaning of queries and content.
This leads to the conclusion that semantic search is not simply about applying NLP and adding synonyms to an index.
It’s true, tokenization does require some real-world knowledge about language construction, and synonyms apply understanding of conceptual matches.
However, they lack, in most cases, an artificial intelligence that is required for search to rise to the level of semantic.
Powered By Vector Search
It is this last bit that makes semantic search both powerful and difficult.
Generally, with the term semantic search, there is an implicit understanding that there is some level of machine learning involved.
Almost as often, this also involves vector search.
Vector search works by encoding details about an item into vectors and then comparing vectors to determine which are most similar.
Again, even a simple example can help.
Take two phrases: “Toyota Prius” and “steak.”
And now let’s compare those to “hybrid.”
Which of the first two are more similar?
Neither would match textually, but you probably would say that “Toyota Prius” is the more similar of the two.
You can say this because you know that a “Prius” is a type of hybrid vehicle because you have seen “Toyota Prius” in a similar context as the word hybrid, such as “Toyota Prius is a hybrid worth considering,” or “hybrid vehicles like the Toyota Prius.”
You’re pretty sure, however, you’ve never seen “steak” and ”hybrid” in such close quarters.
Plotting Vectors To Find Similarity
This is generally how vector search works as well.
A machine learning model takes thousands or millions of examples from the web, books, or other sources and uses this information to then make predictions.
Of course, it is not feasible for the model to go through comparisons one-by-one ( “Are Toyota Prius and hybrid seen together often? How about hybrid and steak?”) and so what happens instead is that the models will encode patterns that it notices about the different phrases.
It’s similar to how you might look at a phrase and say, “this one is positive” or “that one includes a color.”
Except in machine learning the language model doesn’t work so transparently (which is also why language models can be difficult to debug).
These encodings are stored in a vector or a long list of numeric values.
Then, vector search uses math to calculate how similar different vectors are.
Another way to think about the similarity measurements that vector search does is to imagine the vectors plotted out.
This is mind-blowingly difficult if you try to think of a vector plotted into hundreds of dimensions.
If you instead imagine a vector plotted into three dimensions, the principle is the same.
These vectors form a line when plotted, and the question is: which of these lines are closest to each other?
The lines for “steak” and “beef” will be closer than the lines for “steak” and “car” , and so are more similar.
This principle is called a vector, or cosine, similarity.
Vector similarity has a lot of applications.
It can make recommendations based on the previously purchased products, find the most similar image, and can determine which items best match semantically when compared to a user’s query.
Conclusion
Semantic search is a powerful tool for search applications that have come to the forefront with the rise of powerful deep learning models and the hardware to support them.
While we’ve touched on a number of different common applications here, there are even more that use vector search and AI.
Even image search or extracting metadata from images can fall under semantic search.
We’re in exciting times!
And, yet, its application is still early and its known powerfulness can lend itself to a misappropriation of the term.
There are many components in a semantic search pipeline, and getting each one correct is important.
When done correctly, semantic search will use real-world knowledge, especially through machine learning and vector similarity, to match a user query to the corresponding content.
More resources:
Featured Image: magic pictures/Shutterstock
SEO
Content Pruning: Why It Works, and How to Do It
Content pruning sounds pretty appealing: delete a ton of content and see your organic traffic improve. But pruning has risks (like deleting useful pages and useful backlinks), and benefits are not guaranteed: So how does pruning actually work? And when…
SEO
8 Free SEO Reporting Tools
There’s no shortage of SEO reporting tools to choose from—but what are the core tools you need to put together an SEO report?
In this article, I’ll share eight of my favorite SEO reporting tools to help you create a comprehensive SEO report for free.
Price: Free
Google Search Console, often called GSC, is one of the most widely used tools to track important SEO metrics from Google Search.
Most common reporting use case
GSC has a ton of data to dive into, but the main performance indicator SEOs look at first in GSC is Clicks on the main Overview dashboard.
As the data is from Google, SEOs consider it to be a good barometer for tracking organic search performance. As well as clicks data, you can also track the following from the Performance report:
- Total Impressions
- Average CTR
- Average Position
Tip
But for most SEO reporting, GSC clicks data is exported into a spreadsheet and turned into a chart to visualize year-over-year performance.
Favorite feature
One of my favorite reports in GSC is the Indexing report. It’s useful for SEO reporting because you can share the indexed to non-indexed pages ratio in your SEO report.
If the website has a lot of non-indexed pages, then it’s worth reviewing the pages to understand why they haven’t been indexed.
Price: Free
Google Looker Studio (GLS), previously known as Google Data Studio (GDS), is a free tool that helps visualize data in shareable dashboards.
Most common reporting use case
Dashboards are an important part of SEO reporting, and GLS allows you to get a total view of search performance from multiple sources through its integrations.
Out of the box, GLS allows you to connect to many different data sources.
Such as:
- Marketing products – Google Ads, Google Analytics, Display & Video 360, Search Ads 360
- Consumer products – Google Sheets, YouTube, and Google Search Console
- Databases – BigQuery, MySQL, and PostgreSQL
- Social media platforms – Facebook, Reddit, and Twitter
- Files – CSV file upload and Google Cloud Storage
Sidenote.
If you don’t have the time to create your own report manually, Ahrefs has three Google Looker Studio connectors that can help you create automated SEO reporting for any website in a few clicks
Here’s what a dashboard in GLS looks like:
With this type of dashboard, you share reports that are easy to understand with clients or other stakeholders.
Favorite feature
The ability to blend and filter data from different sources, like GA and GSC, means you can get a customized overview of your total search performance, tailored to your website.
Price: Free for 500 URLs
Screaming Frog is a website crawler that helps you audit your website.
Screaming Frog’s free version of its crawler is perfect if you want to run a quick audit on a bunch of URLs. The free version is limited to 500 URLs—making it ideal for crawling smaller websites.
Most common reporting use case
When it comes to reporting, the Reports menu in Screaming Frog SEO Spider has a wealth of information you can look over that covers all the technical aspects of your website, such as analyzing, redirects, canonicals, pagination, hreflang, structured data, and more.
Once you’ve crawled your site, it’s just a matter of downloading the reports you need and working out the main issues to summarize in your SEO report.
Favorite feature
Screaming Frog can pull in data from other tools, including Ahrefs, using APIs.
If you already had access to a few SEO tools’ APIs, you could pull data from all of them directly into Screaming Frog. This is useful if you want to combine crawl data with performance data or other 3rd party tools.
Even if you’ve never configured an API, connecting other tools to Screaming Frog is straightforward.
Price: Free
Ahrefs has a large selection of free SEO tools to help you at every stage of your SEO campaign, and many of these can be used to provide insights for your SEO reporting.
For example, you could use our:
Most common reporting use case
One of our most popular free SEO tools is Ahrefs Webmaster Tools (AWT), which you can use for your SEO reporting.
With AWT, you can:
- Monitor your SEO health over time by setting up scheduled SEO audits
- See the performance of your website
- Check all known backlinks for your website
Favorite feature
Of all the Ahrefs free tools, my favorite is AWT. Within it, site auditing is my favorite feature—once you’ve set it up, it’s a completely hands-free way to keep track of your website’s technical performance and monitor its health.
If you already have access to Google Search Console, it’s a no-brainer to set up a free AWT account and schedule a technical crawl of your website(s).
Price: Free
Ahrefs’ SEO Toolbar is a free Chrome and Firefox extension useful for diagnosing on-page technical issues and performing quick spot checks on your website’s pages.
Most common reporting use case
For SEO reporting, it’s useful to run an on-page check on your website’s top pages to ensure there aren’t any serious on-page issues.
With the free version, you get the following features:
- On-page SEO report
- Redirect tracer with HTTP Headers
- Outgoing links report with link highlighter and broken link checker
- SERP positions
- Country changer for SERP
The SEO toolbar is excellent for spot-checking issues with pages on your website. If you are not confident with inspecting the code, it can also give you valuable pointers on what elements you need to include on your pages to make them search-friendly.
If anything is wrong with the page, the toolbar highlights it, with red indicating a critical issue.
Favorite feature
The section I use the most frequently in the SEO toolbar is the Indexability tab. In this section, you can see whether the page can be crawled and indexed by Google.
Although you can do this by inspecting the code manually, using the toolbar is much faster.
Price: Free
Like GSC, Google Analytics is another tool you can use to track the performance of your website, tracking sessions and conversions and much more on your website.
Most common reporting use case
GA gives you a total view of website traffic from several different sources, such as direct, social, organic, paid traffic, and more.
Favorite feature
You can create and track up to 300 events and 30 conversions with GA4. Previously, with universal analytics, you could only track 20 conversions. This makes conversion and event tracking easier within GA4.
Price: Free
Google Slides is Google’s version of Microsoft PowerPoint. If you don’t have a dashboard set up to report on your SEO performance, the next best thing is to assemble a slide deck.
Many SEO agencies present their report through dashboard insights and PowerPoint presentations. However, if you don’t have access to PowerPoint, then Google Slides is an excellent (free) alternative.
Most common reporting use cases
The most common use of Google Slides is to create a monthly SEO report. If you don’t know what to include in a monthly report, use our SEO report template.
Favorite feature
One of my favorite features is the ability to share your presentation on a video chat directly from Google Slides. You can do this by clicking the camera icon in the top right.
This is useful if you are working with remote clients and makes sharing your reports easy.
Price: Free
Google Trends allows you to view a keyword’s popularity over time in any country. The data shown is the relative popularity ratio scaled from 0-100, not the direct volume of search queries.
Most common reporting use cases
Google Trends is useful for showing how the popularity of certain searches can increase or decrease over time. If you work with a website that often has trending products, services, or news, it can be useful to illustrate this visually in your SEO report.
Google Trends makes it easy to spot seasonal trends for product categories. For example, people want to buy BBQs when the weather is sunny.
Using Google Trends, we can see that peak demand for BBQs usually happens in June-July every year.
Using this data across the last five years, we could be fairly sure when the BBQ season would start and end.
Favorite feature
Comparing two or more search terms against each other over time is one of my favorite uses of Google Trends, as it can be used to tell its own story.
Embellishing your report with trends data allows you to gain further insights into market trends.
You can even dig into trends at a regional level if you need to.
Final thoughts
These free tools will help you put together the foundations for a well-rounded SEO report.
The tools you use for SEO reporting don’t always have to be expensive—even large companies use many of the free tools mentioned to create insights for their client’s SEO reports.
Got more questions? Ping me on X 🙂
SEO
Study Reveals Potential Disruption For Brands & SEO
A new study by Authoritas suggests that Google’s AI-powered Search Generative Experience (SGE), currently being tested with a limited group of users, could adversely impact brand visibility and organic search traffic.
These findings include:
- When an SGE box is expanded, the top organic result drops by over 1,200 pixels on average, significantly reducing visibility.
- 62% of SGE links come from domains outside the top 10 organic results.
- Ecommerce, electronics, and fashion-related searches saw the greatest disruption, though all verticals were somewhat impacted.
Adapting to generative search may require a shift in SEO strategies, focusing more on long-form content, expert insights, and multimedia formats.
As Google continues to invest in AI-powered search, the Authoritas study provides an early look at the potential challenges and opportunities ahead.
High Penetration Rate & Industry-Wide Effects
The study analyzed 2,900 brand and product-related keywords across 15 industry verticals and found that Google displays SGE results for 91.4% of all search queries.
The prevalence of SGE results indicates they impact a majority of websites across various industries.
The research analyzed the typical composition of SGE results. On average, each SGE element contained between 10-11 links sourced from an average of four different domains.
This indicates brands may need to earn multiple links and listings within these AI-curated results to maintain visibility and traffic.
The research also suggests that larger, well-established websites like Quora and Reddit will likely perform better in SGE results than smaller websites and lesser-known brands.
Shifting Dynamics In Organic Search Results
With SGE results occupying the entire first page, websites that currently hold the top positions may experience a significant decrease in traffic and click-through rates.
When a user clicks to expand the SGE element, the study found that, on average, the #1 ranked organic result drops a sizeable 1,255 pixels down the page.
Even if a website ranks number one in organic search, it may effectively be pushed down to the second page due to the prominence of SGE results.
New Competition From Unexpected Sources
The study revealed that SGE frequently surfaces links and content from websites that didn’t appear in the top organic rankings.
On average, only 20.1% of SGE links exactly matched a URL from the first page of Google search results.
An additional 17.9% of SGE links were from the same domains as page one results but linked to different pages. The remaining 62% of SGE links came from sources outside the top organic results.
Challenges For Brand Term Optimization & Local Search
The study reveals that SGE results for branded terms may include competitors’ websites alongside the brand’s own site, potentially leading to increased competition for brand visibility.
Laurence O’Toole, CEO and founder of Authoritas, states:
“Brands are not immune. These new types of generative results introduce more opportunities for third-party sites and even competitors to rank for your brand terms and related brand and product terms that you care about.”
Additionally, local businesses may face similar challenges, as SGE results could feature competing local brands even when users search for a specific brand in a regional context.
Methodology & Limitations
To arrive at these insights, Authoritas analyzed a robust dataset of 2,900 search keywords across a spectrum of query types, including specific brand names, brand + generic terms, brand + product names, generic terms, and specific product names. The keywords were distributed across 15 industry verticals.
The study utilized a consistent desktop browser viewport to quantify pixel-based changes in the search results. Authoritas also developed proprietary “alignment scores” to measure the degree of overlap between traditional organic search results and the new SGE links.
While acknowledging some limitations, such as the keyword set needing to be fully representative of each vertical and the still-evolving nature of SGE, Authoritas maintains that the insights hold value in preparing brands for the new realities of an AI-powered search ecosystem.
Why We Care
The findings of the Authoritas study have implications for businesses, marketers, and SEO professionals. As Google’s SGE becomes more prevalent, it could disrupt traditional organic search rankings and traffic patterns.
Brands that have invested heavily in SEO and have achieved top rankings for key terms may find their visibility and click-through rates diminished by the prominence of SGE results.
SGE introduces new competition from unexpected sources, as most SGE links come from domains outside the top 10 organic results. This means businesses may need to compete not only with their traditional rivals but also with a broader range of websites that gain visibility through SGE.
As Google is a primary source of traffic and leads for many businesses, any changes to its search results can impact visibility, brand awareness, and revenue.
How This Could Help You
While the rise of SGE presents challenges, it also offers opportunities.
Taking into account what we’ve learned from the Authoritas study, here are some actionable takeaways:
- As SGE favors in-depth, informative content, businesses may benefit from investing in comprehensive, well-researched articles and guides that provide value to users.
- Incorporating expert quotes, interviews, and authoritative sources within your content could increase the likelihood of being featured in SGE results.
- Enriching your content with images, videos, and other multimedia elements may help capture the attention of both users and the SGE algorithm.
- Building a strong brand presence across multiple channels, including social media, industry forums, and relevant websites, can increase your chances of appearing in SGE.
- Creating a trustworthy brand and managing your online reputation will be crucial, as SGE may feature competitors alongside your website.
Looking Ahead
While the long-term impact of SGE will depend on user adoption and the perceived usefulness of results, this study’s findings serve as a valuable starting point for businesses and SEO professionals.
By proactively addressing the challenges and opportunities SGE presents, you can increase your chances of success in the new search environment.
Featured Image: BestForBest/Shutterstock
-
SEO6 days ago
Contact Us Page Examples: 44 Designs For Inspiration
-
SEO6 days ago
Google’s Advice For Ranking: Stop Showing
-
SEARCHENGINES5 days ago
Daily Search Forum Recap: March 22, 2024
-
WORDPRESS6 days ago
WordPress Block Themes Explained in 250 Seconds – WordPress.com News
-
PPC6 days ago
The 8 Best Lead Generation Ideas from Marketing Experts
-
SEO6 days ago
Save Time With Keywords Explorer Tool
-
MARKETING7 days ago
Local Search Developments from Q1 2024
-
PPC5 days ago
Mastering Lead Generation in Paid Search Advertising
You must be logged in to post a comment Login