Connect with us

SEO

Everything You Need To Know

Published

on

Everything You Need To Know

Crawling and indexing websites is the first step in a complex process of understanding what webpages are about in order to present them as answers to user queries.

Search engines are constantly improving how they crawl and index websites.

Understanding how Google and Bing approach the task of crawling and indexing websites is useful in developing strategies for improving search visibility.

How Search Engines Work Today: Indexing

Let’s look at the nuts and bolts of how search engines operate.

This article focuses on indexing. So, let’s dive in…

Advertisement

Indexing

Indexing is where the ranking process begins after a website has been crawled.

Indexing essentially refers to the adding of a webpage’s content into Google to be considered for rankings.

When you create a new page on your site, there are a number of ways it can be indexed.

The simplest method of getting a page indexed is to do absolutely nothing.

Google has crawlers following links and thus, provided your site is in the index already and that the new content is linked to from within your site, Google will eventually discover it and add it to its index. More on this later.

How To Get A Page Indexed Faster

But, what if you want Googlebot to get to your page faster?

Advertisement

This can be important if you have timely content or if you’ve made an important change to a page you need Google to know about.

I use faster methods when I’ve optimized a critical page or I’ve adjusted the title and/or description to improve click-throughs. I want to know specifically when they were picked up and displayed in the SERPs to know where the measurement of improvement starts.

In these instances, there are a few additional methods you can use.

1. XML Sitemaps

XML sitemaps are the oldest and a generally reliable way to call a search engine’s attention to content.

An XML sitemap gives search engines a list of all the pages on your site, as well as additional details about it, such as when it was last modified.

A sitemap can be submitted to Bing via Bing Webmaster Tools and it can also be submitted to Google via Search Console.

Advertisement

Definitely recommended!

But when you need a page indexed immediately, it’s not particularly reliable.

2. Request Indexing With Google Search Console

In Search Console, you can “Request Indexing.”

You begin by clicking on the top search field which reads by default, “Inspect and URL in domain.com.”

Enter the URL you want to be indexed, then hit Enter.

If the page is already known to Google, you will be presented with a bunch of information on it. We won’t get into that here but I recommend logging in and seeing what’s there if you haven’t already.

Advertisement

The important button, for our purposes here, appears whether the page has been indexed or not – meaning that it’s good for content discovery or just requesting Google to understand a recent change.

You’ll find the button as shown below.

Screenshot from Google Search Console, October 2019

Within a few seconds to a few minutes, you can search the new content or URL in Google and find the change or new content picked up.

3. Participate In Bing’s IndexNow

Bing has an open protocol that is based on a push method of alerting search engines of new or updated content.

This new search engine indexing protocol is called, IndexNow.

It’s called a push protocol because the idea is to alert search engines using IndexNow about new or updated content which will cause them to come and index it.

An example of a pull protocol is the old XML Sitemap way that depends on a search engine crawler to decide to visit and index it (or to be fetched by Search Console).

Advertisement

The benefit of IndexNow is that it wastes less web hosting and data center resources, which is not only environmentally friendly but it saves on bandwidth resources.

The biggest benefit, however, is faster content indexing.

IndexNow is currently used only by Bing and Yandex.

Implementing IndexNow is easy:

4. Bing Webmaster Tools

In addition to participating in IndexNow, consider a Bing Webmaster Tools account.

If you don’t have a Bing Webmaster Tools account, I can’t recommend it enough.

Advertisement

The info provided within is substantial and will help you better assess problem areas and improve your rankings on Bing, Google, and anywhere else – and probably provide a better user experience as well.

But for getting your content indexed you simply need to click: Configure My Site > Submit URLs.

From there you enter the URL(s) you want indexes and click “Submit.”

bing-webmaster-submit-urlsScreenshot from Bing Webmaster Tools, October 2019

So, that’s almost everything that you need to know about indexing and how search engines do it (with an eye towards where things are going).

More details at the Bing Webmaster Tools URL Submission Tool help page.

There is also a Bing Webmaster Tools Indexing API that can also speed up the time that content appears in Bing’s search results to within hours.  More information about the Bing Indexing API here.

Crawl Budget

We can’t really talk about indexing without talking about the crawl budget.

Advertisement

Basically, crawl budget is a term used to describe the amount of resources that Google will expend crawling a website.

The budget assigned is based on a combination of factors, the two central ones being:

  • How fast your server is (i.e., how much can Google crawl without degrading your user experience).
  • How important your site is.

If you run a major news site with constantly updating content that search engine users will want to be aware of your site will get crawled frequently (dare I say… constantly).

If you run a small barbershop, have a couple of dozen links, and rightfully are not deemed important in this context (you may be an important barber in the area but you’re not important when it comes to the crawl budget), then the budget will be low.

You can read more about crawl budgets and how they’re determined in Google’s explanation here.

Google Has Two Kinds Of Crawling

Indexing by Google begins with crawling, which has two kinds.

The first kind of crawling is Discovery, where Google discovers new webpages to add to the index.

Advertisement

The second kind of crawling is Refresh, where Google finds changes in webpages that are already indexed.

Discover How Search Engines Work

Optimizing websites for search engines begins with good content and it ends with sending it off to get indexed.

Whether you do that with an XML sitemap, Google Search Console URL Submission Tool, Bing Webmaster Tools, or IndexNow, getting that content indexed is the moment where your webpage begins its journey to the top of the search results (if everything works out!).

That’s why it’s important to understand how search indexing works.

How Search Engines Work tackles how search engines function and the key factors that influence search engine results pages.

Download it here.

Advertisement



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Click to comment

You must be logged in to post a comment Login

Leave a Reply

SEO

Google Declares It The “Gemini Era” As Revenue Grows 15%

Published

on

By

A person holding a smartphone displaying the Google Gemini Era logo, with a blurred background of stock market charts.

Alphabet Inc., Google’s parent company, announced its first quarter 2024 financial results today.

While Google reported double-digit growth in key revenue areas, the focus was on its AI developments, dubbed the “Gemini era” by CEO Sundar Pichai.

The Numbers: 15% Revenue Growth, Operating Margins Expand

Alphabet reported Q1 revenues of $80.5 billion, a 15% increase year-over-year, exceeding Wall Street’s projections.

Net income was $23.7 billion, with diluted earnings per share of $1.89. Operating margins expanded to 32%, up from 25% in the prior year.

Ruth Porat, Alphabet’s President and CFO, stated:

Advertisement

“Our strong financial results reflect revenue strength across the company and ongoing efforts to durably reengineer our cost base.”

Google’s core advertising units, such as Search and YouTube, drove growth. Google advertising revenues hit $61.7 billion for the quarter.

The Cloud division also maintained momentum, with revenues of $9.6 billion, up 28% year-over-year.

Pichai highlighted that YouTube and Cloud are expected to exit 2024 at a combined $100 billion annual revenue run rate.

Generative AI Integration in Search

Google experimented with AI-powered features in Search Labs before recently introducing AI overviews into the main search results page.

Regarding the gradual rollout, Pichai states:

“We are being measured in how we do this, focusing on areas where gen AI can improve the Search experience, while also prioritizing traffic to websites and merchants.”

Pichai reports that Google’s generative AI features have answered over a billion queries already:

Advertisement

“We’ve already served billions of queries with our generative AI features. It’s enabling people to access new information, to ask questions in new ways, and to ask more complex questions.”

Google reports increased Search usage and user satisfaction among those interacting with the new AI overview results.

The company also highlighted its “Circle to Search” feature on Android, which allows users to circle objects on their screen or in videos to get instant AI-powered answers via Google Lens.

Reorganizing For The “Gemini Era”

As part of the AI roadmap, Alphabet is consolidating all teams building AI models under the Google DeepMind umbrella.

Pichai revealed that, through hardware and software improvements, the company has reduced machine costs associated with its generative AI search results by 80% over the past year.

He states:

“Our data centers are some of the most high-performing, secure, reliable and efficient in the world. We’ve developed new AI models and algorithms that are more than one hundred times more efficient than they were 18 months ago.

How Will Google Make Money With AI?

Alphabet sees opportunities to monetize AI through its advertising products, Cloud offerings, and subscription services.

Advertisement

Google is integrating Gemini into ad products like Performance Max. The company’s Cloud division is bringing “the best of Google AI” to enterprise customers worldwide.

Google One, the company’s subscription service, surpassed 100 million paid subscribers in Q1 and introduced a new premium plan featuring advanced generative AI capabilities powered by Gemini models.

Future Outlook

Pichai outlined six key advantages positioning Alphabet to lead the “next wave of AI innovation”:

  1. Research leadership in AI breakthroughs like the multimodal Gemini model
  2. Robust AI infrastructure and custom TPU chips
  3. Integrating generative AI into Search to enhance the user experience
  4. A global product footprint reaching billions
  5. Streamlined teams and improved execution velocity
  6. Multiple revenue streams to monetize AI through advertising and cloud

With upcoming events like Google I/O and Google Marketing Live, the company is expected to share further updates on its AI initiatives and product roadmap.


Featured Image: Sergei Elagin/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

brightonSEO Live Blog

Published

on

brightonSEO Live Blog

Hello everyone. It’s April again, so I’m back in Brighton for another two days of sun, sea, and SEO!

Being the introvert I am, my idea of fun isn’t hanging around our booth all day explaining we’ve run out of t-shirts (seriously, you need to be fast if you want swag!). So I decided to do something useful and live-blog the event instead.

Follow below for talk takeaways and (very) mildly humorous commentary. 

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Further Postpones Third-Party Cookie Deprecation In Chrome

Published

on

By

Close-up of a document with a grid and a red stamp that reads "delayed" over the word "status" due to Chrome's deprecation of third-party cookies.

Google has again delayed its plan to phase out third-party cookies in the Chrome web browser. The latest postponement comes after ongoing challenges in reconciling feedback from industry stakeholders and regulators.

The announcement was made in Google and the UK’s Competition and Markets Authority (CMA) joint quarterly report on the Privacy Sandbox initiative, scheduled for release on April 26.

Chrome’s Third-Party Cookie Phaseout Pushed To 2025

Google states it “will not complete third-party cookie deprecation during the second half of Q4” this year as planned.

Instead, the tech giant aims to begin deprecating third-party cookies in Chrome “starting early next year,” assuming an agreement can be reached with the CMA and the UK’s Information Commissioner’s Office (ICO).

The statement reads:

Advertisement

“We recognize that there are ongoing challenges related to reconciling divergent feedback from the industry, regulators and developers, and will continue to engage closely with the entire ecosystem. It’s also critical that the CMA has sufficient time to review all evidence, including results from industry tests, which the CMA has asked market participants to provide by the end of June.”

Continued Engagement With Regulators

Google reiterated its commitment to “engaging closely with the CMA and ICO” throughout the process and hopes to conclude discussions this year.

This marks the third delay to Google’s plan to deprecate third-party cookies, initially aiming for a Q3 2023 phaseout before pushing it back to late 2024.

The postponements reflect the challenges in transitioning away from cross-site user tracking while balancing privacy and advertiser interests.

Transition Period & Impact

In January, Chrome began restricting third-party cookie access for 1% of users globally. This percentage was expected to gradually increase until 100% of users were covered by Q3 2024.

However, the latest delay gives websites and services more time to migrate away from third-party cookie dependencies through Google’s limited “deprecation trials” program.

The trials offer temporary cookie access extensions until December 27, 2024, for non-advertising use cases that can demonstrate direct user impact and functional breakage.

Advertisement

While easing the transition, the trials have strict eligibility rules. Advertising-related services are ineligible, and origins matching known ad-related domains are rejected.

Google states the program aims to address functional issues rather than relieve general data collection inconveniences.

Publisher & Advertiser Implications

The repeated delays highlight the potential disruption for digital publishers and advertisers relying on third-party cookie tracking.

Industry groups have raised concerns that restricting cross-site tracking could push websites toward more opaque privacy-invasive practices.

However, privacy advocates view the phaseout as crucial in preventing covert user profiling across the web.

With the latest postponement, all parties have more time to prepare for the eventual loss of third-party cookies and adopt Google’s proposed Privacy Sandbox APIs as replacements.

Advertisement

Featured Image: Novikov Aleksey/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS