SEO

What It Is & How to Use It

Published

2 years ago

March 28, 2022

Diagnosing technical issues on your website can be one of the most time-consuming but important aspects of running a website.

To make things worse, Google only allows you to inspect one URL at a time to diagnose potential issues on your website (this is done within Google Search Console).

Luckily, there is now a faster way to test your website: enter the Google Search Console URL Inspection API…

What is the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API is a way to bulk-check the data that Google Search Console has on URLs. Its purpose is to help developers and SEOs more efficiently debug and optimize their pages using Google’s own data.

Here’s an example of me using the API to check whether a few URLs are indexed and submitted in my sitemap:

In table form, API showing Chris' URLs are indexed but not submitted in sitemap

What type of data can you get from the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API allows you to pull a wide range of data. Below is a list of some of the most discussed features:

lastCrawlTime

With this field, you can understand exactly when Googlebot last crawled your website. This is extremely useful for SEOs and developers to measure the frequency of Google’s crawling of their sites. Previously, you could only get access to this type of data through log file analysis or spot-checking individual URLs with Google Search Console.

robotsTxtState

With this field, you can understand whether you have any robots.txt rules that will block Googlebot. This is something you can check manually, but being able to test it at scale with Google’s own data is a fantastic step forward.

googleCanonical and userCanonical

In some situations, Google has been known to select a different canonical from the one that has been specified in the code. In this situation, having the ability to compare both (side by side and at scale) using the API is useful for enabling you to make the appropriate changes.

crawledAs

This field allows you to understand which user agent is used for a crawl of your site: Mobile/Desktop. The response codes are below for reference:

DESKTOP – Desktop user agent
MOBILE – Mobile user agent

pageFetchState

Understanding the pageFetchState can help you diagnose server errors, not found 4xxs, soft 404s, redirection errors, pages blocked by robots.txt, and invalid URLs. A list of responses is below for reference.

Field	What it means
PAGE_FETCH_STATE_UNSPECIFIED	Unknown fetch state
SUCCESSFUL	Successful fetch
SOFT_404	Soft 404
BLOCKED_ROBOTS_TXT	Blocked by robots.txt
NOT_FOUND	Not found (404)
ACCESS_DENIED	Blocked due to unauthorized request (401)
SERVER_ERROR	Server error (5xx)
REDIRECT_ERROR	Redirection error
ACCESS_FORBIDDEN	Blocked due to access forbidden (403)
BLOCKED_4XX	Blocked due to other 4xx issue (not 403, 404)
INTERNAL_CRAWL_ERROR	Internal error
INVALID_URL	Invalid URL

indexingState

The indexing state tells you the current status of indexation for the URLs. Apart from the more obvious Pass and Fail responses, there are other responses:

NEUTRAL is equivalent to the “Excluded” message in Search Console.
PARTIAL is equivalent to the “Valid with warnings” message in Search Console.
VERDICT_UNSPECIFIED means that Google is unable to come to a conclusion about the URL(s) in question.

coverageState

This gives you detail on whether a URL has been submitted in your sitemap and indexed.

referringUrls

This allows you to see where each page is linked from, according to Google.

Sitemap

This enables you to understand which URLs are included in the sitemap(s).

Other uses for the API

You can also use the API to inspect your AMP site—if you have one.

How to use the Google Search Console URL Inspection API step by step

Using the Google Search Console URL Inspection API involves making a request to Google. The request parameters you need to define are the URL that you want to inspect and also the URL of the property in Google Search Console.

The request body contains data with the following structure:

If you are curious to learn more about how to use the API, Google has extensive documentation about this.

Below is an example of the type of response you can get from the API:

If you’re not comfortable with code or just want to give it a go straight away, you can use valentin.app’s free Google Bulk Inspect URLs tool. The tool provides a quick way to query the API without any coding skills!

Here’s how to use it. You can:

Go to https://valentin.app/inspect.html, authorize access to your Google account, and select the Search Console property you want to test. Then paste your URLs into the box below. (The data will be processed in your browser and not uploaded to a server or shared with anyone.)
Click the “Inspect URLs” button. The data will start to pull from the API.
Export the data as a CSV or Excel file by clicking the button.
Analyze the data and check for any potential issues.

How can you use the Google Search Console URL Inspection API in practice?

In theory, the Google Search Console URL Inspection API seems like a great way to understand more about your website. However, you can pull so much data that it’s difficult to know where to start. So let’s look at a few examples of use cases.

1. Site migration – diagnosing any technical issues

Site migrations can cause all kinds of issues. For example, developers can accidentally block Google from crawling your site or certain pages via robots.txt.

Luckily, the Google Search Console URL Inspection API makes auditing for these issues a doddle.

For example, you can check whether you’re blocking Googlebot from crawling URLs in bulk by calling robotsTxtState.

Here is an example of me using the Google Search Console URL Inspection API (via valentin.app) to call robotsTxtState to see the current status of my URLs.

In table form, API showing status of Chris' URLs as "allowed"

As you can see, these pages are not blocked by robots.txt, and there are no issues here.

TIP

Site migrations can sometimes lead to unforeseen technical SEO issues. After the migration, we recommend using a tool, such as Ahrefs’ Site Audit, to check your website for over 100 pre-defined SEO issues.

2. Understand if Google has respected your declared canonicals

If you make a change to the canonical tags across your site, you will want to know whether or not Google is respecting them.

You may be wondering why Google ignores the canonical that you declared. Google can do this for a number of reasons, for example:

Your declared canonical is not https. (Google prefers https for canonicals.)
Google has chosen a page that it believes is a better canonical page than your declared canonical.
Your declared canonical is a noindex page.

Below is an example of me using the Google Search Console URL Inspection API to see whether Google has respected my declared canonicals:

In table form, API showing Google respects Chris' canonicals

As we can see from the above screenshot, there are no issues with these particular pages and Google is respecting the canonicals.

TIP

To quickly see if the googleCanonical matches the userCanonical, export the data from the Google Bulk Inspect URLs tool to CSV and use an IF formula in Excel. For example, assuming your googleCanonical data is in Column A and your userCanonical is in Column B, you can use the formula =IF(A2=B2, “Self referencing”,”Non Self Referencing”) to check for non-matching canonicals.

3. Understand when Google recrawls after you make changes to your site

When you update many pages on your website, you will want to know the impact of your efforts. This can only happen after Google has recrawled your site.

With the Google Search Console URL Inspection API, you can see the precise time Google crawled your pages by using lastCrawlTime.

If you can’t get access to the log files for your website, then this is a great alternative to understand how Google crawls your site.

Here’s an example of me checking this:

In table form, API showing "last crawl" date and time for each URL

As you can see in the screenshot above, lastCrawlTime shows the date and time my website was crawled. In this example, the most recent crawl by Google is the homepage.

Understanding when Google recrawls your website following any changes will allow you to link whether or not the changes you made have any positive or negative impact following Google’s crawl.

FAQs

How to get around the Google Search Console URL Inspection API limits?

Although the Google Search Console URL Inspection API is limited to 2,000 queries per day, this query limit is determined by Google Property.

This means you can have multiple properties within one website if they are verified separately in Google Search Console, effectively allowing you to bypass the limit of 2,000 queries per day.

Google Search Console allows you to have 1,000 properties in your Google Search Console account, so this should be more than enough for most users.

Can I use the Google Search Console URL Inspection API on any website?

Another potential limiting factor is you can only run the Google Search Console URL Inspection API on a property that you own in Google Search Console. If you don’t have access to the property, then you cannot audit it using the Google Search Console URL Inspection API.

So this means auditing a site that you don’t have access to can be problematic.

How accurate is the data?

Accuracy of the data itself has been an issue for Google over the last few years. This API gives you access to that data. So arguably, the Google Search Console URL Inspection API is only as good as the data within it.

As we have previously shown in our study of Google Keyword Planner’s accuracy, data from Google is often not as accurate as people assume it to be.

Final thoughts

The Google Search Console URL Inspection API is a great way for site owners to get bulk data directly from Google on a larger scale than what was previously possible from Google Search Console.

Daniel Waisberg and the team behind the Google Search Console URL Inspection API have definitely done a great job of getting this released into the wild.

But one of the criticisms of the Google Search Console URL Inspection API from the SEO community is that the query rate limit is too low for larger sites. (It is capped at 2,000 queries per day, per property.)

For larger sites, this is not enough. Also, despite the possible workarounds, this number still seems to be on the low side.

What’s your experience of using the Google Search Console URL Inspection API? Got more questions? Ping me on Twitter. 🙂

Source link

SEO

4 Ways To Try The New Model From Mistral AI

Published

8 hours ago

December 11, 2023

Max

4 Ways To Try The New Model From Mistral AI

In a significant leap in large language model (LLM) development, Mistral AI announced the release of its newest model, Mixtral-8x7B.

magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%https://t.co/uV4WVdtpwZ%3A6969%2Fannounce&tr=http%3A%2F%https://t.co/g0m9cEUz0T%3A80%2Fannounce

RELEASE a6bbd9affe0c2725c1b7410d66833e24

— Mistral AI (@MistralAI) December 8, 2023

What Is Mixtral-8x7B?

Mixtral-8x7B from Mistral AI is a Mixture of Experts (MoE) model designed to enhance how machines understand and generate text.

Imagine it as a team of specialized experts, each skilled in a different area, working together to handle various types of information and tasks.

A report published in June reportedly shed light on the intricacies of OpenAI’s GPT-4, highlighting that it employs a similar approach to MoE, utilizing 16 experts, each with around 111 billion parameters, and routes two experts per forward pass to optimize costs.

This approach allows the model to manage diverse and complex data efficiently, making it helpful in creating content, engaging in conversations, or translating languages.

Mixtral-8x7B Performance Metrics

Mistral AI’s new model, Mixtral-8x7B, represents a significant step forward from its previous model, Mistral-7B-v0.1.

It’s designed to understand better and create text, a key feature for anyone looking to use AI for writing or communication tasks.

New open weights LLM from @MistralAI

params.json:
– hidden_dim / dim = 14336/4096 => 3.5X MLP expand
– n_heads / n_kv_heads = 32/8 => 4X multiquery
– “moe” => mixture of experts 8X top 2 👀

Likely related code: https://t.co/yrqRtYhxKR

Oddly absent: an over-rehearsed… https://t.co/8PvqdHz1bR pic.twitter.com/xMDRj3WAVh

— Andrej Karpathy (@karpathy) December 8, 2023

This latest addition to the Mistral family promises to revolutionize the AI landscape with its enhanced performance metrics, as shared by OpenCompass.

Mixtral-8x7B: 4 Ways To Try The New Model From Mistral AI

What makes Mixtral-8x7B stand out is not just its improvement over Mistral AI’s previous version, but the way it measures up to models like Llama2-70B and Qwen-72B.

mixtral-8x7b performance metrics compared to llama 2 open source ai models

It’s like having an assistant who can understand complex ideas and express them clearly.

One of the key strengths of the Mixtral-8x7B is its ability to handle specialized tasks.

For example, it performed exceptionally well in specific tests designed to evaluate AI models, indicating that it’s good at general text understanding and generation and excels in more niche areas.

This makes it a valuable tool for marketing professionals and SEO experts who need AI that can adapt to different content and technical requirements.

The Mixtral-8x7B’s ability to deal with complex math and coding problems also suggests it can be a helpful ally for those working in more technical aspects of SEO, where understanding and solving algorithmic challenges are crucial.

This new model could become a versatile and intelligent partner for a wide range of digital content and strategy needs.

How To Try Mixtral-8x7B: 4 Demos

You can experiment with Mistral AI’s new model, Mixtral-8x7B, to see how it responds to queries and how it performs compared to other open-source models and OpenAI’s GPT-4.

Please note that, like all generative AI content, platforms running this new model may produce inaccurate information or otherwise unintended results.

User feedback for new models like this one will help companies like Mistral AI improve future versions and models.

1. Perplexity Labs Playground

In Perplexity Labs, you can try Mixtral-8x7B along with Meta AI’s Llama 2, Mistral-7b, and Perplexity’s new online LLMs.

In this example, I ask about the model itself and notice that new instructions are added after the initial response to extend the generated content about my query.

Screenshot from Perplexity, December 2023 mixtral-8x7b perplexity labs playground

While the answer looks correct, it begins to repeat itself.

Screenshot from Perplexity Labs, December 2023 mixtral-8x7b errors

The model did provide an over 600-word answer to the question, “What is SEO?”

Again, additional instructions appear as “headers” to seemingly ensure a comprehensive answer.

Screenshot from Perplexity Labs, December 2023 what is seo by mixtral-8x7b

2. Poe

Poe hosts bots for popular LLMs, including OpenAI’s GPT-4 and DALL·E 3, Meta AI’s Llama 2 and Code Llama, Google’s PaLM 2, Anthropic’s Claude-instant and Claude 2, and StableDiffusionXL.

These bots cover a wide spectrum of capabilities, including text, image, and code generation.

The Mixtral-8x7B-Chat bot is operated by Fireworks AI.

Screenshot from Poe, December 2023 poe bot for mixtral-8x7b firebase

It’s worth noting that the Fireworks page specifies it is an “unofficial implementation” that was fine-tuned for chat.

When asked what the best backlinks for SEO are, it provided a valid answer.

Screenshot from Poe, December 2023 mixtral-8x7b poe best backlinks response

Compare this to the response offered by Google Bard.

Mixtral-8x7B: 4 Ways To Try The New Model From Mistral AI

Screenshot from Google Bard, December 2023 Mixtral-8x7B: 4 Ways To Try The New Model From Mistral AI

3. Vercel

Vercel offers a demo of Mixtral-8x7B that allows users to compare responses from popular Anthropic, Cohere, Meta AI, and OpenAI models.

Screenshot from Vercel, December 2023 vercel mixtral-8x7b demo compare gpt-4

It offers an interesting perspective on how each model interprets and responds to user questions.

Screenshot from Vercel, December 2023 mixtral-8x7b vs cohere on best resources for learning seo

Like many LLMs, it does occasionally hallucinate.

Screenshot from Vercel, December 2023 mixtral-8x7b hallucinations

4. Replicate

The mixtral-8x7b-32 demo on Replicate is based on this source code. It is also noted in the README that “Inference is quite inefficient.”

Screenshot from Replicate, December 2023 Mixtral-8x7B: 4 Ways To Try The New Model From Mistral AI

In the example above, Mixtral-8x7B describes itself as a game.

Conclusion

Mistral AI’s latest release sets a new benchmark in the AI field, offering enhanced performance and versatility. But like many LLMs, it can provide inaccurate and unexpected answers.

As AI continues to evolve, models like the Mixtral-8x7B could become integral in shaping advanced AI tools for marketing and business.

Featured image: T. Schneider/Shutterstock

Source link

SEO

OpenAI Investigates ‘Lazy’ GPT-4 Complaints On Google Reviews, X

Published

1 day ago

December 10, 2023

Max

OpenAI, the company that launched ChatGPT a little over a year ago, has recently taken to social media to address concerns regarding the “lazy” performance of GPT-4 on social media and Google Reviews.

Screenshot from X, December 2023 OpenAI Investigates ‘Lazy’ GPT-4 Complaints On Google Reviews, X

This move comes after growing user feedback online, which even includes a one-star review on the company’s Google Reviews.

OpenAI Gives Insight Into Training Chat Models, Performance Evaluations, And A/B Testing

OpenAI, through its @ChatGPTapp Twitter account, detailed the complexities involved in training chat models.

Screenshot from X, December 2023 chatgpt openai a/b testing

The organization highlighted that the process is not a “clean industrial process” and that variations in training runs can lead to noticeable differences in the AI’s personality, creative style, and political bias.

Thorough AI model testing includes offline evaluation metrics and online A/B tests. The final decision to release a new model is based on a data-driven approach to improve the “real” user experience.

OpenAI’s Google Review Score Affected By GPT-4 Performance, Billing Issues

This explanation comes after weeks of user feedback about GPT-4 becoming worse on social media networks like X.

Idk if anyone else has noticed this, but GPT-4 Turbo performance is significantly worse than GPT-4 standard.

I know it’s in preview right now but it’s significantly worse.

— Max Weinbach (@MaxWinebach) November 8, 2023

There has been discussion if GPT-4 has become “lazy” recently. My anecdotal testing suggests it may be true.

I repeated a sequence of old analyses I did with Code Interpreter. GPT-4 still knows what to do, but keeps telling me to do the work. One step is now many & some are odd. pic.twitter.com/OhGAMtd3Zq

— Ethan Mollick (@emollick) November 28, 2023

Complaints also appeared in OpenAI’s community forums.

Screenshot from OpenAI, December 2023 openai community forums gpt-4 user feedback

The experience led one user to leave a one-star rating for OpenAI via Google Reviews. Other complaints regarded accounts, billing, and the artificial nature of AI.

Screenshot from Google Reviews, December 2023 openai google reviews star rating

A recent user on Product Hunt gave OpenAI a rating that also appears to be related to GPT-4 worsening.

Screenshot from Product Hunt, December 2023 openai reviews

GPT-4 isn’t the only issue that local reviewers complain about. On Yelp, OpenAI has a one-star rating for ChatGPT 3.5 performance.

OpenAI is now only 3.8 stars on Google Maps and a dismal 1 star on Yelp!

GPT-4’s degradation has really hurt their rating. Hope the business survives.https://t.co/RF8uJH1WQ5 pic.twitter.com/OghAZLCiVu

— Nate Chan (@nathanwchan) December 9, 2023

The complaint:

Screenshot from Yelp, December 2023 yelp openai chatgpt review

In related OpenAI news, the review with the most likes aligns with recent rumors about a volatile workplace, alleging that OpenAI is a “Cutthroat environment. Not friendly. Toxic workers.”

Screenshot from Google Reviews, December 2023 google review for openai toxic workers

The reviews voted the most helpful on Glassdoor about OpenAI suggested that employee frustration and product development issues stem from the company’s shift in focus on profits.

openai employee review on glassdoor

Screenshots from Glassdoor, December 2023 openai employee reviews

This incident provides a unique outlook on how customer and employee experiences can impact any business through local reviews and business ratings platforms.

Screenshot from Google, December 2023 openai inc google business profile local serps google reviews

Google SGE Highlights Positive Google Reviews

In addition to occasional complaints, Google reviewers acknowledged the revolutionary impact of OpenAI’s technology on various fields.

The most positive review mentions about the company appear in Google SGE (Search Generative Experience).

Screenshot from Google SGE, December 2023 Google SGE response on OpenAI

Conclusion

OpenAI’s recent insights into training chat models and response to public feedback about GPT-4 performance illustrate AI technology’s dynamic and evolving nature and its impact on those who depend on the AI platform.

Especially the people who just received an invitation to join ChatGPT Plus after being waitlisted while OpenAI paused new subscriptions and upgrades. Or those developing GPTs for the upcoming GPT Store launch.

As AI advances, professionals in these fields must remain agile, informed, and responsive to technological developments and the public’s reception of these advancements.

Featured image: Tada Images/Shutterstock

Source link

SEO

ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

Published

2 days ago

December 9, 2023

Max

ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

ChatGPT Plus subscriptions and upgrades remain paused after a surge in demand for new features created outages.

Some users who signed up for the waitlist have received invites to join ChatGPT Plus.

Screenshot from Gmail, December 2023 ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

This has resulted in a few shares of the link that is accessible for everyone. For now.

Found a hack to skip chatGPT plus wait list.

Follow the steps
– login to ChatGPT
– now if you click on upgrade
– Signup for waitlist(may not be necessary)
– now change the URL to https://t.co/4izOdNzarG
– Wallah you are in for payment #ChatGPT4 #hack #GPT4 #GPTPlus pic.twitter.com/J1GizlrOAx

— Ashish Mohite is building Notionpack Capture (@_ashishmohite) December 8, 2023

In addition to the invites, signs that more people are getting access to GPTs include an introductory screen popping up on free ChatGPT accounts.

Screenshot from ChatGPT, December 2023 ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

Unfortunately, they still aren’t accessible without a Plus subscription.

Screenshot from ChatGPT, December 2023 chatgpt plus subscriptions upgrades paused waitlist

You can sign up for the waitlist by clicking on the option to upgrade in the left sidebar of ChatGPT on a desktop browser.

Screenshot from ChatGPT, December 2023 ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

OpenAI also suggests ChatGPT Enterprise for those who need more capabilities, as outlined in the pricing plans below.

Screenshot from OpenAI, December 2023 ChatGPT Plus Upgrades Paused; Waitlisted Users Receive Invites

Why Are ChatGPT Plus Subscriptions Paused?

According to a post on X by OpenAI’s CEO Sam Altman, the recent surge in usage following the DevDay developers conference has led to capacity challenges, resulting in the decision to pause ChatGPT Plus signups.

we are pausing new ChatGPT Plus sign-ups for a bit 🙁

the surge in usage post devday has exceeded our capacity and we want to make sure everyone has a great experience.

you can still sign-up to be notified within the app when subs reopen.

— Sam Altman (@sama) November 15, 2023

The decision to pause new ChatGPT signups follows a week where OpenAI services – including ChatGPT and the API – experienced a series of outages related to high-demand and DDoS attacks.

Demand for ChatGPT Plus resulted in eBay listings supposedly offering one or more months of the premium subscription.

chatgpt plus accounts selling ebay for a premium 🫡🇺🇸 https://t.co/VdN8tuexKM pic.twitter.com/W522NGHsRV

— surya (@sdand) November 15, 2023

When Will ChatGPT Plus Subscriptions Resume?

So far, we don’t have any official word on when ChatGPT Plus subscriptions will resume. We know the GPT Store is set to open early next year after recent boardroom drama led to “unexpected delays.”

Therefore, we hope that OpenAI will onboard waitlisted users in time to try out all of the GPTs created by OpenAI and community builders.

What Are GPTs?

GPTs allow users to create one or more personalized ChatGPT experiences based on a specific set of instructions, knowledge files, and actions.

Search marketers with ChatGPT Plus can try GPTs for helpful content assessment and learning SEO.

Two SEO GPTs I’ve created for assessment + learning 👀👇

1. Content Helpfulness and Quality SEO Analyzer: Assess a page content helpfulness, relevance, and quality for your targeted query based on Google’s guidelines vs your competitors and get tips: https://t.co/LsoP2UhF4N pic.twitter.com/O77MHiqwOq

— Aleyda Solis 🕊️ (@aleyda) November 12, 2023

2. The https://t.co/IFmKxxVDpW SEO Teacher: A friendly SEO expert teacher who will help you to learn SEO using reliable https://t.co/sCZ03C7fzq resources: https://t.co/UrMPUYwblH

I hope they’re helpful 🙌🤩

PS: Love how GPT opens up to SO much opportunity 🤯 pic.twitter.com/yqKozcZTDc

— Aleyda Solis 🕊️ (@aleyda) November 12, 2023

There are also GPTs for analyzing Google Search Console data.

oh wow. I think this GPT works.

Export data from GSC comparing keyword rankings before and after an update and upload it to ChatGPT and it will spit out this scatter plot for you.

It’s an easy way to see if most of your keyword declined or improved.

This site was impacted by… pic.twitter.com/wFGSnonqoZ

— Marie Haynes (@Marie_Haynes) November 9, 2023

And GPTs that will let you chat with analytics data from 20 platforms, including Google Ads, GA4, and Facebook.

Google search has indexed hundreds of public GPTs. According to an alleged list of GPT statistics in a GitHub repository, DALL-E, the top GPT from OpenAI, has received 5,620,981 visits since its launch last month. Included in the top 20 GPTs is Canva, with 291,349 views.

Weighing The Benefits Of The Pause

Ideally, this means that developers working on building GPTs and using the API should encounter fewer issues (like being unable to save GPT drafts).

But it could also mean a temporary decrease in new users of GPTs since they are only available to Plus subscribers – including the ones I tested for learning about ranking factors and gaining insights on E-E-A-T from Google’s Search Quality Rater Guidelines.