SEO

Where We Are Today With Google’s Mobile-First Index

Published

1 year ago

July 10, 2022

Where We Are Today With Google’s Mobile-First Index

Okay, so it’s been a few years now since Google announced the mobile-first index.

Most sites have been moved over to Google’s mobile-first index and it’s no longer a “hot” topic in SEO.

I found a tweet from John Mueller, Google Search Advocate, in 2021 that sums up the lack of focus on this topic the best:

My guess is mobile-first indexing has been ongoing for so many years now that it’s more like a “part of life” :). @maxxeight also has a neat tool for testing at https://t.co/r9gFpp95uO . Most sites are moved over, so I don’t expect giant fluctuations.

— 🐐 John 🐐 (@JohnMu) March 12, 2021

Going with that mentality that mobile-first indexing is a “part of life” (which I wholeheartedly agree with), as an SEO, it is helpful to know some of the history and where we are today.

For instance, since the announcement of the mobile-first index years ago, Google has now also placed emphasis on Page Experience, which is a ranking factor and very much incorporates mobile.

Before we jump into that topic, let’s first get into the beginnings of the mobile-first index and what we know so far.

Then, we’ll get into what Google is looking for in mobile usability, what it means to have an identical experience on mobile and desktop, how you can meet Google’s expectations of mobile-first best practices, and more.

Google’s Mobile-First Indexing

No, There Are Not Two Indexes

Google has stated that there isn’t a separate mobile-first index.

Instead, mobile-first indexing means Google primarily uses the mobile version of the webpage for ranking and indexing purposes.

In 2018, Google explained that with mobile-first indexing, the URL of the mobile-friendly version of your site is indexed.

If your website has separate mobile and desktop URLs, Google shows the mobile URL to mobile users and the desktop URL to desktop users.

Regardless, the indexed content will be the mobile version.

Shifting To The Mobile-First Index

At the end of 2017, Google announced that it would start slowly rolling out mobile-first indexing.

By March 2018, Google stated that they were expanding the rollout and instructed websites to prepare.

Fast forward three years later and not all websites have been switched over to the mobile index.

In June 2020, Google stated that while most websites were set to mobile indexing, there were still many that were not.

Google announced at that point that instead of switching in September 2020, it would delay mobile-first indexing until March 2021.

Google cited a number of issues encountered with sites as a reason for delaying the rollout, including problems with robots meta tags, lazy-loading, blocked assets, primary content, and mobile images and videos.

Eventually, Google removed its own self-imposed deadline in November 2021 explaining that there were still sites that were not yet in the mobile-first index because they weren’t ready to be moved over.

Google went on to say that the lack of readiness was due to several unexpected challenges faced by these websites.

According to Google, “because of these difficulties, we’ve decided to leave the timeline open for the last steps of mobile-first indexing.”

Google also stated that “we currently don’t have a specific final date for the move to mobile-first indexing and want to be thoughtful about the remaining bigger steps in that direction.”

Mobile-First Indexing As The Default For New Websites

If your website was published after July 1, 2019, mobile-first indexing is enabled by default.

Google made this announcement in May 2019 and explained that the change applied to websites that were previously unknown to Google Search.

The announcement went into detail about why Google would make mobile-first indexing the default for new websites.

According to Google, after crawling the web with a smartphone Googlebot over the years, they concluded that new websites are typically ready for this type of crawling.

Mobile Usability And Mobile-First Indexing Are Not Synonyms

In January 2019, Mueller explained that if your content does not pass the mobile usability test, it could still be moved to mobile-first indexing.

Even if Search Console’s “mobile usability” report showed that your site had valid URLs, it didn’t mean those pages were ready for mobile-first indexing.

Mobile usability is “completely separate” from mobile-first indexing, according to Mueller. Consequently, pages could be enabled for mobile-first indexing even if they were not considered usable on a mobile device.

You can hear Mueller’s explanation in the video below, starting at the 41:12 mark:

“So, first off, again mobile usability is completely separate from mobile-first indexing.

A site can or cannot be usable from a mobile point of view, but it can still contain all of the content that we need for mobile-first indexing.

An extreme example, if you take something like a PDF file, then on mobile that would be terrible to navigate. The links will be hard to click, the text will be hard to read.

But all of the text is still there, and we could perfectly index that with mobile-first indexing.

Mobile usability is not the same as mobile-first indexing.”

In summary, mobile-friendliness and mobile-responsive layouts are not mandatory for mobile-first indexing.

Since pages without mobile versions still work on a mobile device, they were eligible for indexing.

The Mobile & Desktop Experiences Should Be The Same

Google added to their mobile-first indexing best practices in January 2020, and the big emphasis was on providing an identical experience on mobile and desktop.

Matt Southern provided a great summarized list of what Google meant by the same experience:

Ensuring Googlebot can access and render mobile and desktop page content and resources.
Making sure the mobile site contains the same content as the desktop site.
Using the same meta robots tags on the mobile and desktop site.
Using the same headings on the mobile site and desktop site.
Making sure the mobile and desktop sites have the same structured data.

Google warns that if you purposefully serve less content on the mobile version of a page than the desktop version, you will likely experience a drop in traffic.

The reason? According to Google, they won’t be able to get as much information from the page as before (when the desktop version was used).

Instead, Google recommends that the primary content on the mobile site be the same as on the desktop site. Google even suggests using the same headings on the mobile version.

To drive this point home, even more, Google mentions in its mobile-indexing documentation that only the content on the mobile site is used in indexing.

Therefore, you should be sure that your mobile site has the same content as your desktop site.

Mueller reiterated this fact during Pubcon Pro Virtual 2020 with the following comment:

“…we’re now almost completely indexing the web using a smart phone Googlebot, which matches a lot more what users would actually see when they search.

And one of the things that we noticed that people are still often confused about is with regards to, like if I only have something on desktop, surely Google will still see that and it will also take into account the mobile content.

But actually, it is the case that we will only index the mobile content in the future.

So when a site is shifted over to mobile first indexing, we will drop everything that’s only on the desktop site. We will essentially ignore that.

…anything that you want to have indexed, it needs to be on the mobile site.”

You can read more about Mueller’s comments here: Google Mobile-First Index – Zero Desktop Content March 2021.

Google’s Mobile-First Indexing Best Practices

Google provides a comprehensive list of best practices for mobile-first indexing “to make sure that your users have the best experience.”

Most of the information Google shares as best practices is not really new.

Instead, the list is a compilation of various recommendations and advice that Google has provided elsewhere over the years.

In addition to the list of recommendations above about creating the same experience on mobile and desktop, other best practices include:

Making sure the error page status is the same on the mobile and desktop sites.
Avoiding fragment URLs in the mobile site.
Making sure the desktop pages have equivalent mobile pages.
Verifying both the mobile and desktop sites in Search Console.
Checking hreflang links on separate mobile URLs.
Making sure the mobile site can handle an increased crawl rate.
Making sure the robot.txt directives are the same on the mobile and desktop sites.

Google offers an entire section focused on suggestions for separate URLs.

The “Troubleshooting” section of the best practices document is also worth checking out.

It includes common errors that can either cause your site to not be ready for mobile-first indexing or could lead to a drop in rankings once your site is enabled.

Note that Mueller explained nothing has changed with mobile-first indexing related to sites with separate mobile URLs using rel-canonical. Mueller recommends keeping the annotations the same.

Google will use the mobile URL as canonical even if the rel-canonical points to the desktop URL.

Mueller created a helpful graphic that shows a “before and after” indexing process for desktop and m-dot URLs.

I occasionally get questions about this, so just to be clear: if you have separate mobile URLs (with rel-alternate / rel-canonical links), with mobile first indexing you *don’t* need to change anything. Keep the same annotations. No changes needed. pic.twitter.com/nGPucxPXWn

— 🐝 johnmu.csv (personal) 🐝 (@JohnMu) January 18, 2021

One last note about best practices.

In Google’s mobile-first indexing best practices documentation, it states, “While it’s not required to have a mobile version of your pages to have your content included in Google’s search results, it is very strongly recommended.”

While it might seem obvious to have a mobile version, I have gotten pushback when speaking about mobile-first.

At one conference, an attendee asked during my session if having a mobile version of the site was necessary if no one was coming from a mobile device.

He kept emphasizing “no one.” My answer? Do it anyway.

Not only does Google very highly recommend it, but visitors, especially repeat visitors, might not be using mobile devices because of the poor experience.

We need to focus not just on getting pages ranked in search results, but also on ensuring that the visitor has a good experience once on the page.

Page Experience Update + Mobile-First

The Page Experience update also needs to be part of the conversation.

The Page Experience update was officially released for mobile devices in 2021 and includes measurement signals regarding how visitors perceive their experience of interacting with your web page.

According to Google, this perception goes beyond just the information value provided on the page. Therefore, Google takes into account loading performance, visual stability, and interactivity of the page, which is known as Core Web Vitals.

Page Experience also looks at mobile-friendliness, HTTPS, and intrusive interstitials, which were already a part of the ranking algorithm.

For instance, mobile-friendliness was announced as a ranking factor in 2015, which led to Mobilegeddon (the industry’s name for the update… not Google’s name).

This factor took into account text readability, spacing of tap targets, and unplayable content.

A year later, Google announced that it was strengthening this ranking factor.

Originally, the mobile-friendly update was meant to apply to mobile search results only, but now with the mobile-first index, it applies overall.

Let’s get back to Core Web Vitals.

Core Web Vitals are factors Google considers important in a user’s overall experience on the webpage, including Largest Contentful Paint (LCP), First Input Delay (FID), and Cumulative Layout Shift (CLS).

Each of these factors contributes to the user experience and is scored as “Good,” “Needs Improvement,” or “Poor.”

Now, let’s see how this relates to mobile-first indexing.

There is a lot of overlap between Core Web Vitals and the mobile-first index because both look at how a page performs on a mobile device.

To tie this together, you can reference one of the mobile-first indexing best practices provided by Google, which is to ensure your mobile site loads fast.

Google offers specific recommendations, including using Google PageSpeed Insights and focusing on the “Speed” section. Note that there are other tools you can use too to test speed, such as GTMetrix and WebPageTest.

Martin Splitt, who works in Google’s Developer Relations, was asked in May 2021 if the Page Experience Update was going to roll out on mobile and desktop pages at the same time.

His response was that it would start with mobile pages first, which it did in August 2021. It would be rolled out on desktop pages in February 2022.

It was also made clear that Google would assess mobile pages separately from desktop pages, meaning there is no aggregate score of mobile and desktop (at least not for now).

You can access both the desktop and mobile Page Experience reports in Google Search Console.

Just as you need to pay attention to the desktop and mobile versions of your site for the mobile-first index, you also need to for the Page Experience update.

Check out Core Web Vitals: A Complete Guide for detailed information about this update and how to implement fixes.

One last note before we move on: When Google scores a page, it will test the speed, stability, and usability of the page version that the user ends up seeing.

Here’s where things get tricky. For Core Web Vitals, if you have an AMP version, Google will use it for page experience scoring (i.e., speed, quality, and usability). The mobile version would not be used.

Yet, the mobile version is what would be crawled for the mobile-first index.

So, to sum it up, the AMP version would be used for Core Web Vitals scoring and the mobile version would be used for mobile-first indexing.

Read Google Mobile-First Indexing and Scoring of Sites with Mobile and AMP Versions for the full explanation from Mueller.

Improve Performance In Google’s Mobile-First Index

Here is a consolidated list of items to check that build on some of the best practices already provided.

1. If You Have Multiple Versions, Make Sure Important Content Is Shown On All

Make sure your important content – including structured data, internal links, images, and so on – is on the mobile version of your website, too.

Google even warns in its mobile-indexing best practices that if you have less content on your mobile page than the desktop page, you will experience some traffic loss when your site is moved to mobile-first indexing,

2. Let Googlebot Access And Render Your Content

Google recommends that you use the same meta robots tags on the mobile site, avoid lazy-loading primary content (Googlebot can’t load content that requires user interaction), and allow Googlebot to crawl your resources.

3. Verify Structured Data

Double-check that your structured data is the same on both the desktop and mobile versions of your website and also ensure the URLs are correct.

4. Improve Mobile Page Speed

Page speed has been a factor to consider for a long time and it is even more important with the mobile-first index and Page Experience update.

Advanced Core Web Vitals: A Technical SEO Guide is packed with how-to advice on identifying and addressing speed-related factors that impact Core Web Vitals and mobile-first indexing.

5. Keep An Eye On Mobile Errors

As with most SEO work, getting a site to perform well in the mobile-first index is not a “one and done” task. You need to be closely monitoring Search Console so that you can identify and fix mobile errors.

Make it a habit to regularly view the “mobile usability” and “Core Web Vitals” reports in Search Console.

Keep Reading: Google’s Changelog On Mobile-First Indexing

The changelog in Google’s mobile-first indexing best practices gives a quick recap of the changes since 2016.

As you can tell, there is a lot to know and keep in mind on mobile-first indexing.

Make sure you are staying on top of best practices and monitoring your website’s performance to succeed in the world of mobile-first indexing.

More Resources:

Featured Image: DisobeyArt/Shutterstock

Source link

Related Topics:Googles Index MobileFirst seo Today

Up Next

Google Analytics 4 – Everything You Need To Know

Don't Miss

13 Must-Have Elements Of An Exceptional Blog Post

Click to comment

You must be logged in to post a comment Login

SEO

State Of Marketing Data Standards In The AI Era [Webinar]

Published

13 hours ago

December 2, 2023

Max

State Of Marketing Data Standards In The AI Era [Webinar]

Claravine and Advertiser Perceptions surveyed 140 marketers and agencies to better understand the impact of data standards on marketing data, and they’re ready to present their findings.

Want to learn how you can mitigate privacy risks and boost ROI through data standards?

Watch this on-demand webinar and learn how companies are addressing new privacy laws, taking advantage of AI, and organizing their data to better capture the campaign data they need, as well as how you can implement these findings in your campaigns.

In this webinar, you will:

Gain a better understanding of how your marketing data management compares to enterprise advertisers.
Get an overview of the current state of data standards and analytics, and how marketers are managing risk while improving the ROI of their programs.
Walk away with tactics and best practices that you can use to improve your marketing data now.

Chris Comstock, Chief Growth Officer at Claravine, will show you the marketing data trends of top advertisers and the potential pitfalls that come with poor data standards.

Learn the key ways to level up your data strategy to pinpoint campaign success.

View the slides below or check out the full webinar for all the details.

Join Us For Our Next Webinar!

SaaS Marketing: Expert Paid Media Tips Backed By $150M In Ad Spend

Join us and learn a unique methodology for growth that has driven massive revenue at a lower cost for hundreds of SaaS brands. We’ll dive into case studies backed by real data from over $150 million in SaaS ad spend per year.

Source link

SEO

GPT Store Set To Launch In 2024 After ‘Unexpected’ Delays

Published

1 day ago

December 2, 2023

Max

GPT Store Set To Launch In 2024 After 'Unexpected' Delays

OpenAI shares its plans for the GPT Store, enhancements to GPT Builder tools, privacy improvements, and updates coming to ChatGPT.

OpenAI has scheduled the launch of the GPT Store for early next year, aligning with its ongoing commitment to developing advanced AI technologies.
The GPT Builder tools have received substantial updates, including a more intuitive configuration interface and improved file handling capabilities.
Anticipation builds for upcoming updates to ChatGPT, highlighting OpenAI’s responsiveness to community feedback and dedication to AI innovation.

Source link

SEO

96.55% of Content Gets No Traffic From Google. Here’s How to Be in the Other 3.45% [New Research for 2023]

Published

2 days ago

December 1, 2023

Entireweb News Bot

96.55% of Content Gets No Traffic From Google. Here's How to Be in the Other 3.45% [New Research for 2023]

It’s no secret that the web is growing by millions, if not billions of pages per day.

Our Content Explorer tool discovers 10 million new pages every 24 hours while being very picky about the pages that qualify for inclusion. The “main” Ahrefs web crawler crawls that number of pages every two minutes.

But how much of this content gets organic traffic from Google?

To find out, we took the entire database from our Content Explorer tool (around 14 billion pages) and studied how many pages get traffic from organic search and why.

How many web pages get organic search traffic?

96.55% of all pages in our index get zero traffic from Google, and 1.94% get between one and ten monthly visits.

Distribution of pages by traffic from Content Explorer

Before we move on to discussing why the vast majority of pages never get any search traffic from Google (and how to avoid being one of them), it’s important to address two discrepancies with the studied data:

~14 billion pages may seem like a huge number, but it’s not the most accurate representation of the entire web. Even compared to the size of Site Explorer’s index of 340.8 billion pages, our sample size for this study is quite small and somewhat biased towards the “quality side of the web.”
Our search traffic numbers are estimates. Even though our database of ~651 million keywords in Site Explorer (where our estimates come from) is arguably the largest database of its kind, it doesn’t contain every possible thing people search for in Google. There’s a chance that some of these pages get search traffic from super long-tail keywords that are not popular enough to make it into our database.

That said, these two “inaccuracies” don’t change much in the grand scheme of things: the vast majority of published pages never rank in Google and never get any search traffic.

But why is this, and how can you be a part of the minority that gets organic search traffic from Google?

Well, there are hundreds of SEO issues that may prevent your pages from ranking well in Google. But if we focus only on the most common scenarios, assuming the page is indexed, there are only three of them.

Reason 1: The topic has no search demand

If nobody is searching for your topic, you won’t get any search traffic—even if you rank #1.

For example, I recently Googled “pull sitemap into google sheets” and clicked the top-ranking page (which solved my problem in seconds, by the way). But if you plug that URL into Ahrefs’ Site Explorer, you’ll see that it gets zero estimated organic search traffic:

The top-ranking page for this topic gets no traffic because there's no search demand

This is because hardly anyone else is searching for this, as data from Keywords Explorer confirms:

Keyword data from Ahrefs' Keywords Explorer confirms that this topic has no search demand

This is why it’s so important to do keyword research. You can’t just assume that people are searching for whatever you want to talk about. You need to check the data.

Our Traffic Potential (TP) metric in Keywords Explorer can help with this. It estimates how much organic search traffic the current top-ranking page for a keyword gets from all the queries it ranks for. This is a good indicator of the total search demand for a topic.

You’ll see this metric for every keyword in Keywords Explorer, and you can even filter for keywords that meet your minimum criteria (e.g., 500+ monthly traffic potential):

Filtering for keywords with Traffic Potential (TP) in Ahrefs' Keywords Explorer

Reason 2: The page has no backlinks

Backlinks are one of Google’s top three ranking factors, so it probably comes as no surprise that there’s a clear correlation between the number of websites linking to a page and its traffic.

Pages with more referring domains get more traffic

Same goes for the correlation between a page’s traffic and keyword rankings:

Pages with more referring domains rank for more keywords

Does any of this data prove that backlinks help you rank higher in Google?

No, because correlation does not imply causation. However, most SEO professionals will tell you that it’s almost impossible to rank on the first page for competitive keywords without backlinks—an observation that aligns with the data above.

The key word there is “competitive.” Plenty of pages get organic traffic while having no backlinks…

… but from what I can tell, almost all of them are about low-competition topics.

For example, this lyrics page for a Neil Young song gets an estimated 162 monthly visits with no backlinks:

Example of a page with traffic but no backlinks, via Ahrefs' Content Explorer

But if we check the keywords it ranks for, they almost all have Keyword Difficulty (KD) scores in the single figures:

Some of the low-difficulty keywords a page without traffic ranks for

It’s the same story for this page selling upholstered headboards:

You might have noticed two other things about these pages:

Neither of them get that much traffic. This is pretty typical. Our index contains ~20 million pages with no referring domains, yet only 2,997 of them get more than 1K search visits per month. That’s roughly 1 in every 6,671 pages with no backlinks.
Both of the sites they’re on have high Domain Rating (DR) scores. This metric shows the relative strength of a website’s backlink profile. Stronger sites like these have more PageRank that they can pass to pages with internal links to help them rank.

Bottom line? If you want your pages to get search traffic, you really only have two options:

Target uncompetitive topics that you can rank for with few or no backlinks.
Target competitive topics and build backlinks to rank.

If you want to find uncompetitive topics, try this:

Enter a topic into Keywords Explorer
Go to the Matching terms report
Set the Keyword Difficulty (KD) filter to max. 20
Set the Lowest DR filter to your site’s DR (this will show you keywords with at least one of the same or lower DR ranking in the top 5)

Filtering for low-competition keywords in Ahrefs' Keywords Explorer

(Remember to keep an eye on the TP column to make sure they have traffic potential.)

To rank for more competitive topics, you’ll need to earn or build high-quality backlinks to your page. If you’re not sure how to do that, start with the guides below. Keep in mind that it’ll be practically impossible to get links unless your content adds something to the conversation.

Reason 3. The page doesn’t match search intent

Google wants to give users the most relevant results for a query. That’s why the top organic results for “best yoga mat” are blog posts with recommendations, not product pages.

It's obviously what searchers want when they search for "best yoga mats"

Basically, Google knows that searchers are in research mode, not buying mode.

It’s also why this page selling yoga mats doesn’t show up, despite it having backlinks from more than six times more websites than any of the top-ranking pages:

Page selling yoga mats that has lots of backlinks

Number of linking websites to the top-ranking pages for "best yoga mats"

Luckily, the page ranks for thousands of other more relevant keywords and gets tens of thousands of monthly organic visits. So it’s not such a big deal that it doesn’t rank for “best yoga mats.”

Number of keyword rankings for the page selling yoga mats

However, if you have pages with lots of backlinks but no organic traffic—and they already target a keyword with traffic potential—another quick SEO win is to re-optimize them for search intent.

We did this in 2018 with our free backlink checker.

It was originally nothing but a boring landing page explaining the benefits of our product and offering a 7-day trial:

Original landing page for our free backlink checker

After analyzing search intent, we soon realized the issue:

People weren’t looking for a landing page, but rather a free tool they could use right away.

So, in September 2018, we created a free tool and published it under the same URL. It ranked #1 pretty much overnight, and has remained there ever since.