Connect with us

SEO

Everything You Need To Know

Published

on

Everything You Need To Know

Crawling and indexing websites is the first step in a complex process of understanding what webpages are about in order to present them as answers to user queries.

Search engines are constantly improving how they crawl and index websites.

Understanding how Google and Bing approach the task of crawling and indexing websites is useful in developing strategies for improving search visibility.

How Search Engines Work Today: Indexing

Let’s look at the nuts and bolts of how search engines operate.

This article focuses on indexing. So, let’s dive in…

Advertisement

Indexing

Indexing is where the ranking process begins after a website has been crawled.

Indexing essentially refers to the adding of a webpage’s content into Google to be considered for rankings.

When you create a new page on your site, there are a number of ways it can be indexed.

The simplest method of getting a page indexed is to do absolutely nothing.

Google has crawlers following links and thus, provided your site is in the index already and that the new content is linked to from within your site, Google will eventually discover it and add it to its index. More on this later.

How To Get A Page Indexed Faster

But, what if you want Googlebot to get to your page faster?

Advertisement

This can be important if you have timely content or if you’ve made an important change to a page you need Google to know about.

I use faster methods when I’ve optimized a critical page or I’ve adjusted the title and/or description to improve click-throughs. I want to know specifically when they were picked up and displayed in the SERPs to know where the measurement of improvement starts.

In these instances, there are a few additional methods you can use.

1. XML Sitemaps

XML sitemaps are the oldest and a generally reliable way to call a search engine’s attention to content.

An XML sitemap gives search engines a list of all the pages on your site, as well as additional details about it, such as when it was last modified.

A sitemap can be submitted to Bing via Bing Webmaster Tools and it can also be submitted to Google via Search Console.

Advertisement

Definitely recommended!

But when you need a page indexed immediately, it’s not particularly reliable.

2. Request Indexing With Google Search Console

In Search Console, you can “Request Indexing.”

You begin by clicking on the top search field which reads by default, “Inspect and URL in domain.com.”

Enter the URL you want to be indexed, then hit Enter.

If the page is already known to Google, you will be presented with a bunch of information on it. We won’t get into that here but I recommend logging in and seeing what’s there if you haven’t already.

Advertisement

The important button, for our purposes here, appears whether the page has been indexed or not – meaning that it’s good for content discovery or just requesting Google to understand a recent change.

You’ll find the button as shown below.

Screenshot from Google Search Console, October 2019

Within a few seconds to a few minutes, you can search the new content or URL in Google and find the change or new content picked up.

3. Participate In Bing’s IndexNow

Bing has an open protocol that is based on a push method of alerting search engines of new or updated content.

This new search engine indexing protocol is called, IndexNow.

It’s called a push protocol because the idea is to alert search engines using IndexNow about new or updated content which will cause them to come and index it.

An example of a pull protocol is the old XML Sitemap way that depends on a search engine crawler to decide to visit and index it (or to be fetched by Search Console).

Advertisement

The benefit of IndexNow is that it wastes less web hosting and data center resources, which is not only environmentally friendly but it saves on bandwidth resources.

The biggest benefit, however, is faster content indexing.

IndexNow is currently used only by Bing and Yandex.

Implementing IndexNow is easy:

4. Bing Webmaster Tools

In addition to participating in IndexNow, consider a Bing Webmaster Tools account.

If you don’t have a Bing Webmaster Tools account, I can’t recommend it enough.

Advertisement

The info provided within is substantial and will help you better assess problem areas and improve your rankings on Bing, Google, and anywhere else – and probably provide a better user experience as well.

But for getting your content indexed you simply need to click: Configure My Site > Submit URLs.

From there you enter the URL(s) you want indexes and click “Submit.”

bing-webmaster-submit-urlsScreenshot from Bing Webmaster Tools, October 2019

So, that’s almost everything that you need to know about indexing and how search engines do it (with an eye towards where things are going).

More details at the Bing Webmaster Tools URL Submission Tool help page.

There is also a Bing Webmaster Tools Indexing API that can also speed up the time that content appears in Bing’s search results to within hours.  More information about the Bing Indexing API here.

Crawl Budget

We can’t really talk about indexing without talking about the crawl budget.

Advertisement

Basically, crawl budget is a term used to describe the amount of resources that Google will expend crawling a website.

The budget assigned is based on a combination of factors, the two central ones being:

  • How fast your server is (i.e., how much can Google crawl without degrading your user experience).
  • How important your site is.

If you run a major news site with constantly updating content that search engine users will want to be aware of your site will get crawled frequently (dare I say… constantly).

If you run a small barbershop, have a couple of dozen links, and rightfully are not deemed important in this context (you may be an important barber in the area but you’re not important when it comes to the crawl budget), then the budget will be low.

You can read more about crawl budgets and how they’re determined in Google’s explanation here.

Google Has Two Kinds Of Crawling

Indexing by Google begins with crawling, which has two kinds.

The first kind of crawling is Discovery, where Google discovers new webpages to add to the index.

Advertisement

The second kind of crawling is Refresh, where Google finds changes in webpages that are already indexed.

Discover How Search Engines Work

Optimizing websites for search engines begins with good content and it ends with sending it off to get indexed.

Whether you do that with an XML sitemap, Google Search Console URL Submission Tool, Bing Webmaster Tools, or IndexNow, getting that content indexed is the moment where your webpage begins its journey to the top of the search results (if everything works out!).

That’s why it’s important to understand how search indexing works.

How Search Engines Work tackles how search engines function and the key factors that influence search engine results pages.

Download it here.

Advertisement



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Click to comment

You must be logged in to post a comment Login

Leave a Reply

SEO

Google March 2024 Core Update Officially Completed A Week Ago

Published

on

By

Graphic depicting the Google logo with colorful segments on a blue circuit board background, accompanied by the text "Google March 2024 Core Update.

Google has officially completed its March 2024 Core Update, ending over a month of ranking volatility across the web.

However, Google didn’t confirm the rollout’s conclusion on its data anomaly page until April 26—a whole week after the update was completed on April 19.

Many in the SEO community had been speculating for days about whether the turbulent update had wrapped up.

The delayed transparency exemplifies Google’s communication issues with publishers and the need for clarity during core updates

Google March 2024 Core Update Timeline & Status

First announced on March 5, the core algorithm update is complete as of April 19. It took 45 days to complete.

Advertisement

Unlike more routine core refreshes, Google warned this one was more complex.

Google’s documentation reads:

“As this is a complex update, the rollout may take up to a month. It’s likely there will be more fluctuations in rankings than with a regular core update, as different systems get fully updated and reinforce each other.”

The aftershocks were tangible, with some websites reporting losses of over 60% of their organic search traffic, according to data from industry observers.

The ripple effects also led to the deindexing of hundreds of sites that were allegedly violating Google’s guidelines.

Addressing Manipulation Attempts

In its official guidance, Google highlighted the criteria it looks for when targeting link spam and manipulation attempts:

  • Creating “low-value content” purely to garner manipulative links and inflate rankings.
  • Links intended to boost sites’ rankings artificially, including manipulative outgoing links.
  • The “repurposing” of expired domains with radically different content to game search visibility.

The updated guidelines warn:

“Any links that are intended to manipulate rankings in Google Search results may be considered link spam. This includes any behavior that manipulates links to your site or outgoing links from your site.”

John Mueller, a Search Advocate at Google, responded to the turbulence by advising publishers not to make rash changes while the core update was ongoing.

Advertisement

However, he suggested sites could proactively fix issues like unnatural paid links.

Mueller stated on Reddit:

“If you have noticed things that are worth improving on your site, I’d go ahead and get things done. The idea is not to make changes just for search engines, right? Your users will be happy if you can make things better even if search engines haven’t updated their view of your site yet.”

Emphasizing Quality Over Links

The core update made notable changes to how Google ranks websites.

Most significantly, Google reduced the importance of links in determining a website’s ranking.

In contrast to the description of links as “an important factor in determining relevancy,” Google’s updated spam policies stripped away the “important” designation, simply calling links “a factor.”

This change aligns with Google’s Gary Illyes’ statements that links aren’t among the top three most influential ranking signals.

Advertisement

Instead, Google is giving more weight to quality, credibility, and substantive content.

Consequently, long-running campaigns favoring low-quality link acquisition and keyword optimizations have been demoted.

With the update complete, SEOs and publishers are left to audit their strategies and websites to ensure alignment with Google’s new perspective on ranking.

Core Update Feedback

Google has opened a ranking feedback form related to this core update.

You can use this form until May 31 to provide feedback to Google’s Search team about any issues noticed after the core update.

While the feedback provided won’t be used to make changes for specific queries or websites, Google says it may help inform general improvements to its search ranking systems for future updates.

Advertisement

Google also updated its help documentation on “Debugging drops in Google Search traffic” to help people understand ranking changes after a core update.


Featured Image: Rohit-Tripathi/Shutterstock

FAQ

After the update, what steps should websites take to align with Google’s new ranking criteria?

After Google’s March 2024 Core Update, websites should:

  • Improve the quality, trustworthiness, and depth of their website content.
  • Stop heavily focusing on getting as many links as possible and prioritize relevant, high-quality links instead.
  • Fix any shady or spam-like SEO tactics on their sites.
  • Carefully review their SEO strategies to ensure they follow Google’s new guidelines.

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Declares It The “Gemini Era” As Revenue Grows 15%

Published

on

By

A person holding a smartphone displaying the Google Gemini Era logo, with a blurred background of stock market charts.

Alphabet Inc., Google’s parent company, announced its first quarter 2024 financial results today.

While Google reported double-digit growth in key revenue areas, the focus was on its AI developments, dubbed the “Gemini era” by CEO Sundar Pichai.

The Numbers: 15% Revenue Growth, Operating Margins Expand

Alphabet reported Q1 revenues of $80.5 billion, a 15% increase year-over-year, exceeding Wall Street’s projections.

Net income was $23.7 billion, with diluted earnings per share of $1.89. Operating margins expanded to 32%, up from 25% in the prior year.

Ruth Porat, Alphabet’s President and CFO, stated:

Advertisement

“Our strong financial results reflect revenue strength across the company and ongoing efforts to durably reengineer our cost base.”

Google’s core advertising units, such as Search and YouTube, drove growth. Google advertising revenues hit $61.7 billion for the quarter.

The Cloud division also maintained momentum, with revenues of $9.6 billion, up 28% year-over-year.

Pichai highlighted that YouTube and Cloud are expected to exit 2024 at a combined $100 billion annual revenue run rate.

Generative AI Integration in Search

Google experimented with AI-powered features in Search Labs before recently introducing AI overviews into the main search results page.

Regarding the gradual rollout, Pichai states:

“We are being measured in how we do this, focusing on areas where gen AI can improve the Search experience, while also prioritizing traffic to websites and merchants.”

Pichai reports that Google’s generative AI features have answered over a billion queries already:

Advertisement

“We’ve already served billions of queries with our generative AI features. It’s enabling people to access new information, to ask questions in new ways, and to ask more complex questions.”

Google reports increased Search usage and user satisfaction among those interacting with the new AI overview results.

The company also highlighted its “Circle to Search” feature on Android, which allows users to circle objects on their screen or in videos to get instant AI-powered answers via Google Lens.

Reorganizing For The “Gemini Era”

As part of the AI roadmap, Alphabet is consolidating all teams building AI models under the Google DeepMind umbrella.

Pichai revealed that, through hardware and software improvements, the company has reduced machine costs associated with its generative AI search results by 80% over the past year.

He states:

“Our data centers are some of the most high-performing, secure, reliable and efficient in the world. We’ve developed new AI models and algorithms that are more than one hundred times more efficient than they were 18 months ago.

How Will Google Make Money With AI?

Alphabet sees opportunities to monetize AI through its advertising products, Cloud offerings, and subscription services.

Advertisement

Google is integrating Gemini into ad products like Performance Max. The company’s Cloud division is bringing “the best of Google AI” to enterprise customers worldwide.

Google One, the company’s subscription service, surpassed 100 million paid subscribers in Q1 and introduced a new premium plan featuring advanced generative AI capabilities powered by Gemini models.

Future Outlook

Pichai outlined six key advantages positioning Alphabet to lead the “next wave of AI innovation”:

  1. Research leadership in AI breakthroughs like the multimodal Gemini model
  2. Robust AI infrastructure and custom TPU chips
  3. Integrating generative AI into Search to enhance the user experience
  4. A global product footprint reaching billions
  5. Streamlined teams and improved execution velocity
  6. Multiple revenue streams to monetize AI through advertising and cloud

With upcoming events like Google I/O and Google Marketing Live, the company is expected to share further updates on its AI initiatives and product roadmap.


Featured Image: Sergei Elagin/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

brightonSEO Live Blog

Published

on

brightonSEO Live Blog

Hello everyone. It’s April again, so I’m back in Brighton for another two days of sun, sea, and SEO!

Being the introvert I am, my idea of fun isn’t hanging around our booth all day explaining we’ve run out of t-shirts (seriously, you need to be fast if you want swag!). So I decided to do something useful and live-blog the event instead.

Follow below for talk takeaways and (very) mildly humorous commentary. 

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS