Connect with us

SEO

What It Is & How to Use It

Published

on

What It Is & How to Use It

Diagnosing technical issues on your website can be one of the most time-consuming but important aspects of running a website.

To make things worse, Google only allows you to inspect one URL at a time to diagnose potential issues on your website (this is done within Google Search Console).

Luckily, there is now a faster way to test your website: enter the Google Search Console URL Inspection API…

What is the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API is a way to bulk-check the data that Google Search Console has on URLs. Its purpose is to help developers and SEOs more efficiently debug and optimize their pages using Google’s own data.

Here’s an example of me using the API to check whether a few URLs are indexed and submitted in my sitemap:

In table form, API showing Chris' URLs are indexed but not submitted in sitemap

What type of data can you get from the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API allows you to pull a wide range of data. Below is a list of some of the most discussed features:

lastCrawlTime

With this field, you can understand exactly when Googlebot last crawled your website. This is extremely useful for SEOs and developers to measure the frequency of Google’s crawling of their sites. Previously, you could only get access to this type of data through log file analysis or spot-checking individual URLs with Google Search Console.

robotsTxtState

With this field, you can understand whether you have any robots.txt rules that will block Googlebot. This is something you can check manually, but being able to test it at scale with Google’s own data is a fantastic step forward.

googleCanonical and userCanonical

In some situations, Google has been known to select a different canonical from the one that has been specified in the code. In this situation, having the ability to compare both (side by side and at scale) using the API is useful for enabling you to make the appropriate changes.

crawledAs

This field allows you to understand which user agent is used for a crawl of your site: Mobile/Desktop. The response codes are below for reference:

  • DESKTOP – Desktop user agent
  • MOBILE – Mobile user agent

pageFetchState

Understanding the pageFetchState can help you diagnose server errors, not found 4xxs, soft 404s, redirection errors, pages blocked by robots.txt, and invalid URLs. A list of responses is below for reference.

Field What it means
PAGE_FETCH_STATE_UNSPECIFIED Unknown fetch state
SUCCESSFUL Successful fetch
SOFT_404 Soft 404
BLOCKED_ROBOTS_TXT Blocked by robots.txt
NOT_FOUND Not found (404)
ACCESS_DENIED Blocked due to unauthorized request (401)
SERVER_ERROR Server error (5xx)
REDIRECT_ERROR Redirection error
ACCESS_FORBIDDEN Blocked due to access forbidden (403)
BLOCKED_4XX Blocked due to other 4xx issue (not 403, 404)
INTERNAL_CRAWL_ERROR Internal error
INVALID_URL Invalid URL

indexingState

The indexing state tells you the current status of indexation for the URLs. Apart from the more obvious Pass and Fail responses, there are other responses:

  • NEUTRAL is equivalent to the “Excluded” message in Search Console.
  • PARTIAL is equivalent to the “Valid with warnings” message in Search Console.
  • VERDICT_UNSPECIFIED means that Google is unable to come to a conclusion about the URL(s) in question.

coverageState 

This gives you detail on whether a URL has been submitted in your sitemap and indexed.

referringUrls

This allows you to see where each page is linked from, according to Google.

Sitemap 

This enables you to understand which URLs are included in the sitemap(s).

Other uses for the API

You can also use the API to inspect your AMP site—if you have one.

How to use the Google Search Console URL Inspection API step by step

Using the Google Search Console URL Inspection API involves making a request to Google. The request parameters you need to define are the URL that you want to inspect and also the URL of the property in Google Search Console.

The request body contains data with the following structure:

Structure of data

If you are curious to learn more about how to use the API, Google has extensive documentation about this.

Below is an example of the type of response you can get from the API:

Code response from API

If you’re not comfortable with code or just want to give it a go straight away, you can use valentin.app’s free Google Bulk Inspect URLs tool. The tool provides a quick way to query the API without any coding skills!

Here’s how to use it. You can:

  1. Go to https://valentin.app/inspect.html, authorize access to your Google account, and select the Search Console property you want to test. Then paste your URLs into the box below. (The data will be processed in your browser and not uploaded to a server or shared with anyone.)
  2. Click the “Inspect URLs” button. The data will start to pull from the API.
  3. Export the data as a CSV or Excel file by clicking the button.
  4. Analyze the data and check for any potential issues.

How can you use the Google Search Console URL Inspection API in practice?

In theory, the Google Search Console URL Inspection API seems like a great way to understand more about your website. However, you can pull so much data that it’s difficult to know where to start. So let’s look at a few examples of use cases.

1. Site migration – diagnosing any technical issues

Site migrations can cause all kinds of issues. For example, developers can accidentally block Google from crawling your site or certain pages via robots.txt.

Luckily, the Google Search Console URL Inspection API makes auditing for these issues a doddle.

For example, you can check whether you’re blocking Googlebot from crawling URLs in bulk by calling robotsTxtState.

Here is an example of me using the Google Search Console URL Inspection API (via valentin.app) to call robotsTxtState to see the current status of my URLs.

In table form, API showing status of Chris' URLs as "allowed"

As you can see, these pages are not blocked by robots.txt, and there are no issues here.

TIP

 Site migrations can sometimes lead to unforeseen technical SEO issues. After the migration, we recommend using a tool, such as Ahrefs’ Site Audit, to check your website for over 100 pre-defined SEO issues.

2. Understand if Google has respected your declared canonicals

If you make a change to the canonical tags across your site, you will want to know whether or not Google is respecting them.

You may be wondering why Google ignores the canonical that you declared. Google can do this for a number of reasons, for example:

  • Your declared canonical is not https. (Google prefers https for canonicals.)
  • Google has chosen a page that it believes is a better canonical page than your declared canonical.
  • Your declared canonical is a noindex page.

Below is an example of me using the Google Search Console URL Inspection API to see whether Google has respected my declared canonicals:

In table form, API showing Google respects Chris' canonicals

As we can see from the above screenshot, there are no issues with these particular pages and Google is respecting the canonicals.

TIP

To quickly see if the googleCanonical matches the userCanonical, export the data from the Google Bulk Inspect URLs tool to CSV and use an IF formula in Excel. For example, assuming your googleCanonical data is in Column A and your userCanonical is in Column B, you can use the formula =IF(A2=B2, “Self referencing”,”Non Self Referencing”) to check for non-matching canonicals.

3. Understand when Google recrawls after you make changes to your site

When you update many pages on your website, you will want to know the impact of your efforts. This can only happen after Google has recrawled your site.

With the Google Search Console URL Inspection API, you can see the precise time Google crawled your pages by using lastCrawlTime.

If you can’t get access to the log files for your website, then this is a great alternative to understand how Google crawls your site.

Here’s an example of me checking this:

In table form, API showing "last crawl" date and time for each URL

As you can see in the screenshot above, lastCrawlTime shows the date and time my website was crawled. In this example, the most recent crawl by Google is the homepage.

Understanding when Google recrawls your website following any changes will allow you to link whether or not the changes you made have any positive or negative impact following Google’s crawl.

FAQs

How to get around the Google Search Console URL Inspection API limits?

Although the Google Search Console URL Inspection API is limited to 2,000 queries per day, this query limit is determined by Google Property.

This means you can have multiple properties within one website if they are verified separately in Google Search Console, effectively allowing you to bypass the limit of 2,000 queries per day.

Google Search Console allows you to have 1,000 properties in your Google Search Console account, so this should be more than enough for most users.

Can I use the Google Search Console URL Inspection API on any website?

Another potential limiting factor is you can only run the Google Search Console URL Inspection API on a property that you own in Google Search Console. If you don’t have access to the property, then you cannot audit it using the Google Search Console URL Inspection API.

So this means auditing a site that you don’t have access to can be problematic.

How accurate is the data?

Accuracy of the data itself has been an issue for Google over the last few years. This API gives you access to that data. So arguably, the Google Search Console URL Inspection API is only as good as the data within it.

Tim Guillot shares tweet to John Mu about data issues; Tim adds the issues are "not fixed"

As we have previously shown in our study of Google Keyword Planner’s accuracy, data from Google is often not as accurate as people assume it to be.

Final thoughts

The Google Search Console URL Inspection API is a great way for site owners to get bulk data directly from Google on a larger scale than what was previously possible from Google Search Console.

Daniel Waisberg and the team behind the Google Search Console URL Inspection API have definitely done a great job of getting this released into the wild.

But one of the criticisms of the Google Search Console URL Inspection API from the SEO community is that the query rate limit is too low for larger sites. (It is capped at 2,000 queries per day, per property.)

For larger sites, this is not enough. Also, despite the possible workarounds, this number still seems to be on the low side.

What’s your experience of using the Google Search Console URL Inspection API? Got more questions? Ping me on Twitter. 🙂




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Click to comment

You must be logged in to post a comment Login

Leave a Reply

SEO

Stop Overcomplicating Things. Entity SEO is Just SEO

Published

on

Stop Overcomplicating Things. Entity SEO is Just SEO

“Entity SEO”.

Sounds scary, doesn’t it? Not only does the word “entity” sound foreign, it feels like yet another thing to add to your never-ending SEO to-do list. You’re barely afloat when it comes to SEO, but ohgawd here comes one more new thing to dedicate your scarce resources.

I have good news for you though: You don’t have to do entity SEO.

Why? Because you’re probably already doing it.

Let’s start from the beginning.

In 2012, Google announced the Knowledge Graph. The Knowledge Graph is a knowledge base of entities and the relationships between them.

An entity is any object or concept that can be distinctly identified. This includes tangibles like people, places, and organizations, and intangibles like colors, concepts, and feelings.

For example, the footballer Federico Chiesa is an entity:

The footballer Federico Chiesa is an entityThe footballer Federico Chiesa is an entity

So is the famous British-Indian restaurant Dishoom:

The British-Indian restaurant Dishoom is an entityThe British-Indian restaurant Dishoom is an entity

Entities are connected by edges, which describe the relationships between them.

Introducing the Knowledge Graph helped improve Google’s search results because:

  • Google could better understand search intent — People search for the same thing but describe it in different ways. Google can now understand this and serve the same results.
  • It reduced reliance on keyword matching — Matching the number of keywords on a page doesn’t guarantee relevance; also it prevents crafty SEOs from keyword stuffing.
  • It reduced Google’s computational load — The Internet is virtually infinite and Google simply cannot understand the meaning of every word, paragraph, webpage, and website. Entities provide a structure where Google can improve understanding while minimizing load.

For example, even though we didn’t mention the actor’s name, Google can understand we’re looking for Harrison Ford and therefore shows his filmography:

Google understands Harrison Ford as an entity and can show us his filmographyGoogle understands Harrison Ford as an entity and can show us his filmography

That’s because Hans Solo and Harrison Ford are closely connected entities in the Knowledge Graph. Google shows Knowledge Graph data in SERP features like Knowledge Panels and Knowledge Cards.

Google shows a knowledge panel for Apple, the technology companyGoogle shows a knowledge panel for Apple, the technology company

With this knowledge, we can then define entity SEO as optimizing your website or webpages for such entities.

If Google has moved to entity-oriented search, then entity SEO is just SEO. As my colleague Patrick Stox says, “The entity identification part is more on Google’s end than on our end.”

I mean, if you look at the ‘entity SEO’ tactics you find in blog posts, you’ll discover that they’re mostly just SEO tactics:

  • Earn a Wikipedia page
  • Create a Google Business Profile
  • Add internal links
  • Create all digital assets Google is representing on the page (e.g., videos, images, Twitter)
  • Develop topical authority
  • Include semantically related words on a page
  • Add schema markup

Let’s be honest. If you’re serious about SEO and are investing in it, then it’s likely you’re already doing most of the above.

Regardless of entities, wouldn’t you want a Wikipedia page? After all, it confers benefits beyond “entity SEO”. Brand recognition, backlinks from one of the world’s most authoritative sites (albeit nofollow)—any company would want that.

If you’re a local business, you’ve probably created a Google Business Profile. Adding internal links is just SEO 101.

And billions of blistering barnacles, creating all digital assets Google wants to see, like images and videos, is practically marketing 101. If you’re a Korean recipe site and want to be associated with the kimchi jjigae entity, wouldn’t you already know you need to make a video and have photos of the cooking process?

Google shows images in the knowledge panel for the entity kimchi jjigaeGoogle shows images in the knowledge panel for the entity kimchi jjigae

When I started my breakdance site years ago, I knew nothing about SEO and content marketing but I still knew I needed to make YouTube videos. Because guess what? It’s hard to learn breakdancing from words. I don’t think I needed an entity SEO to tell me that.

Topical authority is an SEO concept where a website aims to become the go-to authority on one or more topics. Call me crazy, but it feels like blogging 101. Read most guides on how to start a blog and I’m sure you’ll find a subheading called “niche down”. And once you niche down, it’s inevitable you’ll create content surrounding that one topic.

If I start a breakdance site, what are the chances I’ll write about contemporary dance or pop art? Pretty low.

In fact, topical authority is similar to the Wiki Strategy, which Nat Eliason wrote about in 2017. There wasn’t a single mention of entities. It was just the right way to make content for the Internet.

I think the biggest problem here isn’t entities versus keywords or that topical authority is a brand-new strategy. It’s simply that many SEOs are driven by short-sightedness or the wrong incentives.

You can target a whole bunch of unrelated keywords that have high search volume, gain incredible amounts of search traffic, and brag about how successful you are as an SEO.

Some of the pages sending HubSpot the most search traffic has barely anything to do with their core product. A page on how to type the shrug emoji? The most famous quotes?

HubSpot's top pages that sends them the most search trafficHubSpot's top pages that sends them the most search traffic

This is not to single out HubSpot—I’m sure they have their reasons, as explored by Ryan here—but to illustrate that many companies do the exact same thing. And when Google stops rewarding this behavior, all of a sudden companies realise they do need to write about their core competencies. They need to “build topical authority”.

I don’t want to throw the baby out with the bathwater because I do see value in the last two ‘entity SEO tactics’. But again, if you’re doing something similar to the Wiki Strategy for your site, chances are you would have naturally included entities or semantically relevant words without thinking too much about it. It’s difficult to create content about kimchi jjigae without mentioning kimchi, pork, or gochujang.

However, to prevent the curse of knowledge or simply to avoid blindspots, checking for important subtopics you might have missed is useful. At Ahrefs, we run a page-level content gap analysis and look out for subtopics:

Open in Content gap feature in Keywords ExplorerOpen in Content gap feature in Keywords Explorer

For example, if we ran a content gap analysis on “inbound marketing” for the top three ranking pages, we see that we might need to include these subtopics:

  • What is inbound marketing
  • Inbound marketing strategy
  • Inbound marketing examples
  • Inbound marketing tools
Content gap report for inbound marketingContent gap report for inbound marketing

Finally, adding schema markup makes the most sense because it’s how Google recognizes entities and better understands the content of web pages. But if it’s just one new tactic—which I believe is already part of ‘standard’ SEO and you might already be doing it—then there’s no need to create a category to define the “new era” (voice SEO, where art thou?)

Final thoughts

Two years ago, someone on Reddit asked for an SEO workflow that utilized super advanced SEO methodologies:

A question on RedditA question on Reddit

The top answer: None of the above.

Comments on RedditComments on Reddit

When our Chief Marketing Officer Tim Soulo tweeted about this Reddit thread, he got similar replies too:

Replies to Tim Soulo's tweetReplies to Tim Soulo's tweet

And even though I don’t know him, this is a person after my own heart:

A tweet agreeing that entity SEO is a fadA tweet agreeing that entity SEO is a fad

You don’t have to worry about entity SEO. If you have passion for a topic and are creating high-quality content that fulfills what people are looking for, then you’re likely already doing “entity SEO”.

Just follow this meme: Make stuff people like.

Midwit meme showing you just need to make stuff people likeMidwit meme showing you just need to make stuff people like

 

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Assigning The Right Conversion Values To Make Value-Based Bidding Work For Lead Gen

Published

on

By

Assigning The Right Conversion Values To Make Value-Based Bidding Work For Lead Gen

Last week, we tackled setting your data strategy for value-based bidding.

The next key is to assign the right values for the conversion actions that are important to your business.

We know this step is often seen as trickier for lead gen-focused businesses than, say, ecommerce businesses.

How much is a whitepaper download, newsletter signup, or online quote request worth to your business? While you may not have exact figures, that’s OK. What you do know is they aren’t all valued equally.

Check out the quick 2-minute video in our series below, and then keep reading as we dive deeper into assigning conversion values to optimize your value-based bidding strategy.

Understanding Conversion Values

First, let’s get on the same page about what “conversion value” means.

A conversion refers to a desired action taken by a user, such as filling out a lead form, making a purchase, or signing up for a newsletter.

Conversion value is simply a numerical representation of how much each of these conversions is worth to your business.

Estimating The Value Of Each Conversion

Ideally, you’d have a precise understanding of how much revenue each conversion generates.

However, we understand that this is not always feasible.

In such cases, it’s perfectly acceptable to use “proxy values” – estimations that align with your business priorities.

The important thing is to ensure that these proxy values reflect the relative importance of different conversions to your business.

For example, a whitepaper download may indicate less “value” than a product demo registration based on what you understand about your past customer acquisition efforts.

Establishing Proxy Values

Let’s explore some scenarios to illustrate how you might establish proxy values.

Take the event florist example mentioned in the video. You’ve seen that clients who provide larger guest counts or budgets in their online quote requests tend to result in more lucrative events.

Knowing this, you can assign higher proxy values to these leads compared to those with smaller guest counts or budgets.

Similarly, if you’re an auto insurance advertiser, you might leverage your existing lead scoring system as a basis for proxy values. Leads with higher scores, indicating a greater likelihood of a sale, would naturally be assigned higher values.

You don’t need to have exact value figures to make value-based bidding effective. Work with your sales and finance teams to help identify the key factors that influence lead quality and value.

This will help you understand which conversion actions indicate a higher likelihood of becoming a customer – and even which actions indicate the likelihood of becoming a higher-value customer for your business.

Sharing Conversion Values With Google Ads

Once you’ve determined the proxy values for your conversion actions, you’ll need to share that information with Google Ads. This enables the system to prioritize actions that drive the most value for your business.

To do this, go to the Summary tab on the Conversions page (under the Goals icon) in your account. From there, you can edit your conversion actions settings to input the value for each. More here.

As I noted in the last episode, strive for daily uploads of your conversion data, if possible, to ensure Google Ads has the most up-to-date information by connecting your sources via Google Ads Data Manager or the Google Ads API.

Fine-Tuning With Conversion Value Rules

To add another layer of precision, you can utilize conversion value rules.

Conversion value rules allow you to adjust the value assigned to a conversion based on specific attributes or conditions that aren’t already indicated in your account. For example, you may have different margins for different types of customers.

Instead of every lead form submission having the same static value you’ve assigned, you can tell Google Ads which leads are more valuable to your business based on three factors:

  • Location: You might adjust conversion values based on the geographical location of the user. For example, if users in a particular region tend to convert at a higher rate or generate more revenue.
  • Audience: You can tailor conversion values based on specific audience segments, such as first-party data or Google audience lists.
  • Device: Consider adjusting conversion values based on the device the user is using. Perhaps users on mobile devices convert at a higher rate – you could increase their conversion value to reflect that.

When implementing these rules, your value-based bidding strategies (maximize conversion value with an optional target ROAS) will take them into account and optimize accordingly.

Conversion value rules can be set at the account or campaign levels. They are supported in Search, Shopping, Display, and Performance Max campaigns.

Google Ads will prioritize showing your ads to users predicted to be more likely to generate those leads you value more.

Conversion Value Rules And Reporting

These rules also impact how you report conversion value in your account.

For example, you may value a lead at $5, but know that these leads from Californian users are typically worth twice as much. With conversion value rules, you could specify this, and Google Ads would multiply values for users from California by two and report that accordingly in the conversion volume column in your account.

Additionally, you can segment your conversion value rules in Campaigns reporting to see the impact by selecting Conversions, then Value rule adjustment.

There are three segment options:

  • Original value (rule applied): Total original value of conversions, which then had a value rule applied.
  • Original value (no rule applied): Total recorded value of conversions that did not have a value rule applied.
  • Audience, Location, Device, or No Condition: The net adjustment when value rules were applied.

You can add the conversion value rules column to your reporting as well. These columns are called “All value adjustment” and “Value adjustment.”

Also note that reporting for conversion value rules applies to all conversions, not just the ones in the ‘conversions’ column.

Conversion Value Rule Considerations

You can also create more complex rules by combining conditions.

For example, if you observe that users from Texas who have also subscribed to your newsletter are exceptionally valuable, you could create a rule that increases their conversion value even further.

When using conversion value rules, keep in mind:

  • Start Simple: Begin by implementing a few basic conversion value rules based on your most critical lead attributes.
  • Additive Nature of Rules: Conversion value rules are additive. If multiple rules apply to the same user, their effects will be combined.
  • Impact on Reporting: The same adjusted value that’s determined at bidding time is also used for reporting.
  • Regular Review for Adjustment: As your business evolves and you gather more data, revisit your conversion values and rules to ensure they remain aligned with your goals.

Putting The Pieces Together

Assigning the right values to your conversions is a crucial step in maximizing the effectiveness of your value-based bidding strategies.

By providing Google Ads with accurate and nuanced conversion data, you empower the system to make smarter decisions, optimize your bids, and ultimately drive more valuable outcomes for your business.

Up next, we’ll talk about determining which bid strategy is right for you. Stay tuned!

More resources: 


Featured Image: BestForBest/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Expert Embedding Techniques for SEO Success

Published

on

By

Expert Embedding Techniques for SEO Success

AI Overviews are here, and they’re making a big impact in the world of SEO. Are you up to speed on how to maximize their impact?

Watch on-demand as we dive into the fascinating world of Google AI Overviews and their functionality, exploring the concept of embeddings and demystifying the complex processes behind them.

We covered which measures play a crucial role in how Google AI assesses the relevance of different pieces of content, helping to rank and select the most pertinent information for AI-generated responses.

You’ll see:

  • An understanding of the technical side of embeddings & how they work, enabling efficient information retrieval and comparison.
  • Insights into AI Content curation, including the criteria and algorithms used to rank and choose the most relevant snippets for AI-generated overviews.
  • A visualization of the step-by-step process of how AI overviews are constructed, with a clear perspective on the decision-making process behind AI-generated content.

With Scott Stouffer from Market Brew, we explored their AI Overviews Visualizer, a tool that deconstructs AI Overviews and provides an inside look at how Snippets and AI Overviews are curated. 

If you’re looking to clarify misconceptions around AI, or looking to face the challenge of optimizing your own content for the AI Overview revolution, then be sure to watch this webinar.

View the slides below, or check out the full presentation for all the details.

Join Us For Our Next Webinar!

[Expert Panel] How Agencies Leverage AI Tools To Drive ROI

Join us as we discuss the importance of AI to your performance as an agency or small business, and how you can use it successfully.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending