While Google has made strides in understanding user intent, Director & Product Manager Elizabeth Tucker says two types of queries remain challenging.

In a recent episode of Google’s Search Off The Record podcast, Tucker discussed some lingering pain points in the company’s efforts to match users with the information they seek.

Among the top offenders were searches containing the word “not” and queries involving prepositions, Tucker reveals:

“Prepositions, in general, are another hard one. And one of the really big, exciting breakthroughs was the BERT paper and transformer-based machine learning models when we started to be able to get some of these complicated linguistic issues right in searches.”

BERT, or Bidirectional Encoder Representations from Transformers, is a neural network-based technique for natural language processing that Google began leveraging in search in 2019.

The technology is designed to understand the nuances and context of words in searches rather than treating queries as a bag of individual terms.

‘Not’ There Yet

Despite the promise of BERT and similar advancements, Tucker acknowledged that Google’s ability to parse complex queries is still a work in progress.

Searches with the word “not” remain a thorn in the search engine’s side, Tucker explains:

“It’s really hard to know when ‘not’ means that you don’t want the word there or when it has a different kind of semantic meaning.”

For example, Google’s algorithms could interpret a search like “shoes not made in China” in multiple ways.

Does the user want shoes made in countries other than China, or are they looking for information on why some shoe brands have moved their manufacturing out of China?

This ambiguity poses a challenge for websites trying to rank for such queries. If Google can’t match the searcher’s intent with the content on a page, it may struggle to surface the most relevant results.

The Preposition Problem

Another area where Google’s algorithms can stumble is prepositions, which show the relationship between words in a sentence.

Queries like “restaurants with outdoor seating” or “hotels near the beach” rely on prepositions to convey key information about the user’s needs.

For SEO professionals, this means that optimizing for queries with prepositions may require some extra finesse.

It’s not enough to include the right keywords on a page; the content needs to be structured to communicate the relationships between those keywords.

The Long Tail Challenge

The difficulties Google faces with complex queries are particularly relevant to long-tail searches—those highly specific, often multi-word phrases that make up a significant portion of all search traffic.

Long-tail keywords are often seen as a golden opportunity for SEO, as they tend to have lower competition and can signal a high level of user intent.

However, if Google can’t understand these complex queries, it may be harder for websites to rank for them, even with well-optimized content.

The Road Ahead

Tucker noted that Google is actively improving its handling of these linguistically challenging queries, but a complete solution may still be a way off.

Tucker said:

“I would not say this is a solved problem. We’re still working on it.”

In the meantime, users may need to rephrase their searches or try different query formulations to find the information they’re looking for – a frustrating reality in an age when many have come to expect Google to understand their needs intuitively.

Why SEJ Cares

While BERT and similar advancements have helped Google understand user intent, the search giant’s struggles with “not” queries and prepositions remind us that there’s still plenty of room for improvement.

As Google continues to invest in natural language processing and other AI-driven technologies, it remains to be seen how long these stumbling blocks will hold back the search experience.

What It Means For SEO

So, what can SEO professionals and website owners do in light of this information? Here are a few things to keep in mind:

1. Focus on clarity and specificity in your content. The more you can communicate the relationships between key concepts and phrases, the easier it will be for Google to understand and rank your pages.
2. Use structured data and other technical SEO best practices to help search engines parse your content more effectively.
3. Monitor your search traffic and rankings for complex queries, and be prepared to adjust your strategy if you see drops or inconsistencies.
4. Monitor Google’s efforts to improve its natural language understanding and be ready to adapt as new algorithms and technologies emerge.

Listen to the full podcast episode below:

In a recent LinkedIn post, Google Analyst Gary Illyes raised awareness about two issues plaguing web crawlers: soft 404 and other “crypto” errors.

These seemingly innocuous mistakes can negatively affect SEO efforts.

Understanding Soft 404s

Soft 404 errors occur when a web server returns a standard “200 OK” HTTP status code for pages that don’t exist or contain error messages. This misleads web crawlers, causing them to waste resources on non-existent or unhelpful content.

Illyes likened the experience to visiting a coffee shop where every item is unavailable despite being listed on the menu. While this scenario might be frustrating for human customers, it poses a more serious problem for web crawlers.

As Illyes explains:

“Crawlers use the status codes to interpret whether a fetch was successful, even if the contents of the page is basically just an error message. They might happily go back to the same page again and again wasting your resources, and if there are many such pages, exponentially more resources.”

The Hidden Costs Of Soft Errors

The consequences of soft 404 errors extend beyond the inefficient use of crawler resources.

According to Illyes, these pages are unlikely to appear in search results because they are filtered out during indexing.

To combat this issue, Illyes advises serving the appropriate HTTP status code when the server or client encounters an error.

This allows crawlers to understand the situation and allocate their resources more effectively.

Illyes also cautioned against rate-limiting crawlers with messages like “TOO MANY REQUESTS SLOW DOWN,” as crawlers cannot interpret such text-based instructions.

Why SEJ Cares

Soft 404 errors can impact a website’s crawlability and indexing.

By addressing these issues, crawlers can focus on fetching and indexing pages with valuable content, potentially improving the site’s visibility in search results.

Eliminating soft 404 errors can also lead to more efficient use of server resources, as crawlers won’t waste bandwidth repeatedly visiting error pages.

How This Can Help You

To identify and resolve soft 404 errors on your website, consider the following steps:

1. Regularly monitor your website’s crawl reports and logs to identify pages returning HTTP 200 status codes despite containing error messages.
2. Implement proper error handling on your server to ensure that error pages are served with the appropriate HTTP status codes (e.g., 404 for not found, 410 for permanently removed).
3. Use tools like Google Search Console to monitor your site’s coverage and identify any pages flagged as soft 404 errors.

Proactively addressing soft 404 errors can improve your website’s crawlability, indexing, and SEO.

Featured Image: Julia Tim/Shutterstock

SEO agencies obsess over their SEO reporting process. It’s their main method to share their achievements with their clients. Without it, clients could be left in the dark about their SEO progress—and trust me, you don’t want that.

In this article, I’ll share the structures of some real-world agency reports that have been shared with me and how different size agencies approach SEO reporting.

SEO agencies juggle multiple clients, so time spent on a fixed task like reporting can quickly add up.

For example, let’s say your agency has five clients and spends two hours per month on the entire SEO reporting process.

That’s over one day per month just spent on SEO reporting.

For this reason, as an agency owner, you want your clients to be high-paying with standardized reporting deliverables, but this is often far from reality.

And it can often look something like this:

And often, the higher the budget, the more tailored your reporting becomes.

Let’s face it: An enterprise client probably won’t be impressed by a basic PDF report you generated in 10 seconds using a third-party tool if they’re paying $XX,000 per month.

Likewise, a client paying $1000 per month would probably not know where to start if you gave them a 60-page SEO report and hooked them up with a Tableau dashboard.

So, based on this, we can say that there are two main types of client reports:

• Small-medium business (SMB) SEO reports
• Enterprise businesses’ SEO reports

But before we discuss the details, let’s explore the main differences between SMB and enterprise SEO reporting.

In my experience, SEO reporting for SMBs usually consists of three elements:

• SEO dashboards – Allows the client to check their SEO performance or KPIs at any time—usually fully automated
• SEO report – Periodic update on the total SEO campaign, usually monthly. Focuses on commentary and insights, and the format of the report can vary
• Presentation – Often, a video call with the client to run through the report and get feedback on the SEO performance

Agencies can automate their SEO reporting easily by using a free and easy-to-use solution like Google Looker Studio (GLS).

With Google Looker Studio, there are three options:

If you don’t want to create your own dashboards, we’ve done the hard work for you and have three Google Looker Studio connectors that pull the best bits from Ahrefs.

If you aren’t confident with APIs, this is one of the best ways to get data out of Ahrefs so your clients can see it without time-consuming manual reporting.

SEO reports for SMBs are usually a document that gives a periodic update on a website’s performance.

So what’s normally included in an SMB SEO report? In my experience, it can cover some or all of the following topics—depending on the focus of the client.

Not sure how to tweak it? Here are some real SEO agency report structure examples you can take inspiration from.

SEO agency report structure #1

With this report a lot of the detail was in the organic visibility section, where it split out visibility by sector.

Also interestingly, it detailed the links acquired during the month for the agency in a classic link report—at the enterprise level, this is less common as enterprise clients acquire links all the time without lifting a finger.

SEO agency report structure #2

This is a delivery-focused report. The “impact analysis” section is interesting as it details the impact that their landing page optimization work has had, concluding with two growth figures.

This is a good way to communicate to the client the value of the work you are doing.

Clients like to have their say when it comes to the SEO calls—after all, they’re paying for it. For SMBs, ensuring they get a good ROI is usually top of their minds.

At an SMB level, the agency should be able to provide a comprehensive assessment of the SEO status of the website and get feedback from the client.

For most SMB clients, the usual format is this:

• Weekly status update call
• Monthly report call following delivery of the monthly report

In my experience, it rarely deviates from this format.

When it comes to dashboard reporting, enterprise clients will usually expect a Tableau, PowerBI, or a custom-built solution, plus some data from Google Looker Studio.

Here’s a snapshot of what that can look like:

I used Tableau when I was working with enterprise clients and found it hugely useful for SEO reporting.

The deliverables for enterprise SEO reports are broadly the same as those for SMB reports, but as always, the devil is in the details.

In short, there’s usually:

• More personalization to the client’s business
• More tools used – Rather than having one or two trusted tools for SEO reporting, a “big six” agency will have access to most, if not all, of the best enterprise SEO tools in the market
• More reports created and shared with different teams

Here are three examples of enterprise SEO reporting for inspiration.

Media agency report structure #1

This report example shows that it’s mostly focused on performance and technical SEO. This agency report had a separate content report that they shared with the content team.

Media agency report structure #2

In this example, the client already had Tableau and Google Looker Studio dashboards set up and got most of their data from these two sources.

The report was created to communicate SEO activity within the business and educate stakeholders about its value.

As you can see, the focus of this report was organic performance, technical SEO, competitor performance, and dissuading clients from self-sabotage (AMP).

Another thing to notice: there is no executive summary.

The client just wanted to drop straight into the organic performance, and this was a screenshot directly from a Tableau dashboard with commentary.

This client operated in ~20 major international markets and needed a summary of the most important movements within those markets.

This report was mainly used to educate other stakeholders on SEO’s benefits and gain buy-in for further SEO improvements to the website. The “industry updates” section helped to do this.

Media agency report structure #3

Although reporting with a spreadsheet may seem archaic to some, it was what this enterprise SEO client had always done—and they weren’t prepared to change.

In this example, the spreadsheet report was a weekly requirement, and the client also requested a monthly and quarterly report in slide format.

Presenting your reports to enterprise clients is a big deal, and you’ll often present your findings to more than one team during the month.

Here are three examples of the reporting presentation schedules that I had with some of my previous clients. As you can see, a single report or dashboard is rarely enough for most enterprise SEO clients.

Client 1

Client 2

Client 3

Final thoughts

Got more questions? Ping me on X. 🙂