Connect with us

SEO

Everything We Know About IndexNow So Far

Published

on

Everything We Know About IndexNow So Far

Hopefully, you’ve heard about IndexNow at this point.

We covered its announcement in October and in December, I had Fabrice Canel, Principal Program Manager for Microsoft Bing, on the SEJ Show to talk about this revolution in how search engines discover new and updated content.

But what is it, exactly, and what do you need to know about it for SEO?

Here’s what we know so far.

Why We Need IndexNow

Search engines have been discovering content the same way for a long time.

Advertisement

Way back in 1993, JumpStation became the first internet discovery tool to send out bots to follow (or crawl) links from website to website and create a content map, or index.

Within the next few years, other search engines adopted this process as a way to find new content.

This process was included in an algorithm that also considered content, relevancy, and other factors – and search engine optimization as we know it was born.

It didn’t take long before SEO-savvy web publishers recognized the importance of indexing in securing high search engine rankings. In fact, without indexed content, a website won’t generate organic traffic.

But therein lies the rub: Unless you’re a large, highly trafficked website, it may be days or even weeks before a search engine crawls your site and indexes new pages. Heck, even if you are a large ecommerce site or marketplace, it may take engines weeks to pick up new arrivals or out of stock products as well – especially when depending on discoverability with basic crawls.

Meanwhile, you may have lost important traffic to competitors.

Advertisement

To streamline this process, Microsoft Bing released a new initiative in October 2021 called IndexNow.

So What Is IndexNow?

IndexNow is an open-source protocol that allows website publishers to instantly index across participating search engines, updating results based upon the latest content changes.

Simply put, it’s a simple ping that notifies search engines that a URL and its content have been added, updated, or deleted.

By eliminating the need for exploratory crawls, IndexNow seeks to improve overall internet efficiency.

How Does IndexNow Work?

There are two different ways search engines obtain index data: Pull and push.

Pull indexing occurs when a search engine visits your site to request web pages and gathers data from the server.

Advertisement

This is how search engines traditionally operate.

Push indexing is when the web publisher or content management system notifies the search engine that pages have been added or deleted or other changes have been made.

That’s what’s truly groundbreaking about the IndexNow – it allows all URLs submitted to any IndexNow-enabled search engine to be simultaneously submitted to all the other search engines using the protocol.

You can read more about the technical aspects of how IndexNow works here.

Which Search Engines Use IndexNow?

At the time of publication, Bing and Yandex are using IndexNow.

A handful of others are reportedly testing the protocol, according to an unnamed expert.

Advertisement

So far, DuckDuckGo has not announced any plans to adopt IndexNow.

However, it’s worth noting that the privacy-focused search engine draws from up to 400 sources in addition to using its own DuckDuckBot web crawler in the compilation of its search results.

Given that Bing and Yandex are among those sources, you could see updated content appearing in DuckDuckGo quicker due to IndexNow, as well.

Even Bing’s largest competitor, Google, the foremost of all search engines, have confirmed they will be testing the IndexNow protocol.

And while they haven’t gone so far as announce whether or not they will adopt it, the fact they are experimenting with it shows a recognition of its capabilities.

Google’s top concerns about IndexNow seem to be related to sustainability and efficiency.

Advertisement

Google is currently using HTTP/2, an efficient fundamental data transfer protocol for more than half of all crawls, and may ultimately decide this is a better fit for their needs.

Or they may decide to build their own alternative if they determine an API-based approach is needed – and it’s not a foregone conclusion that they will.

But not everyone is on board.

There are plenty of people in the SEO community who don’t see the need for this protocol, instead claiming XML sitemaps already accomplish what IndexNow is trying to do.

You can read more about the debate here.

How Do You Use IndexNow?

Notifying search engines of updates to your web content using IndexNow is a straightforward process:

Advertisement
  1. Generate an API Key – This is submitted alongside URLs to ensure ownership of the domain. You can do this using an online key generator tool like the one found here.
  2. Host API Key – Your API key is hosted on the root directory in txt format.
  3. Submit URLs with Parameters – You can either submit URLs individually or in bulk. Submit your key location as a parameter.
  4. Check Your Submissions – Using the Bing Webmaster Tools portal, you can verify which URLs have been submitted and discovered.

For examples and more in-depth instructions, you can visit the IndexNow site.

More advanced users can use Python for bulk indexation and URL submission automation.

This requires a basic understanding of Python syntax and familiarity with Python libraries and packages.

We have a handy step-by-step guide to walk you through this process here.

Which Tools Sync With IndexNow?

Any website developer can use the process outlined here to take advantage of IndexNow, but what’s really exciting is that you don’t have to know HTML from HDMI to reap the benefits.

A number of large companies, including LinkedIn, MSN, and GitHub are all planning migrations to IndexNow.

Additionally, other search solution providers like Botify, OnCrawl, Onely and Yext are in the process of adopting this protocol.

Advertisement

Content management systems (CMS) are also jumping on board.

WordPress already offers an IndexNow plugin. Duda supports this proactive approach to web crawling, and Wix plans to integrate soon.

Bing recently announced Rank Math and All In One SEO integration for IndexNow, as well.

Content delivery networks (CDN) supporting IndexNow include Cloudflare and Akamai.

Through their proxy servers, this will speed up data delivery and bring it closer to users.

What Are The Benefits Of Using IndexNow?

In addition to emphasizing the evolution of indexing from pull to push, the primary benefit of IndexNow for publishers is the elimination of the time between updates being made and search engines discovering them.

Advertisement

By allowing webmasters to notify all participating URLs with a single API call, search engine content discovery is streamlined, which in turn makes the web more efficient.

This is advantageous to site owners, as growing requirements are reduced on the server.

Search engine spiders no longer have to perform exploratory crawls to determine whether a page has been updated.

This enables servers to work more efficiently without the need for a redundant site load, which in turn reduces energy usage.

And unlike an XML sitemap, which only redirects SEO crawlers to pages within your website, IndexNow allows you to inform search engines of non-200 status code pages.

IndexNow Frequently Asked Questions

What Happens After I Submit A URL?

IndexNow conveys your changes to search engines, which in turn apply them to page rankings.

Advertisement

Using IndexNow doesn’t guarantee web pages will be crawled or index and it may take time for changes to be reflected.

How Many URLs Can I submit Using IndexNow?

You can provide search engines with a list of up to 10,000 URLs with a single API call.

Will I Rank Better In Search Results If I Index My Content Faster?

Yes, the sooner your pages are indexed, the more quickly they can start competing for high-ranking search engine spots.

What Does All This Mean?

The march of technology is inexorable.

And as such a vital part of the modern world, it’s only natural that changes to the internet, and in particular search engines, would be part of them.

By shifting the process of indexing sites away from a crawling process to something faster and more efficient, IndexNow seems to be on the cusp of a revolution.

Advertisement

And the fact that companies like MSN, GitHub, and WordPress are all joining in seems to hint that this protocol will soon become the norm.

More resources:


Featured Image: ra2 studio/Shutterstock




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

Blog Post Checklist: Check All Prior to Hitting “Publish”

Published

on

Check All Prior to Hitting “Publish” – SEO Consulting

One thing I have clearly realized while being a full-time blogger and editor is that most writers (no matter how much we love them) are terrible at formatting.

I mean, I get it.

Writing is a creative process, and things like formatting, interlinking and proofreading may keep you from letting your creativity flourish.

One workaround is hiring a VA but if you have control issues (like I do) and still want to do everything on your own, here’s a free blog checklist that’s easy to follow and help you publish a perfectly formatted article each time!

A good formatting checklist should be:

Advertisement
  • As short as possible (The longer it is, the more chances there are that none of your contributors will follow all the steps);
  • As concise as possible (Avoid explaining the reason why you want it to be formatted that way. The more you explain, the less obvious your point becomes. Just list your requirements)
  • As easy to understand as possible (Add a few screenshots, bold most important points, etc).

Feel free to make yourself a copy of this checklist, edit minor details (like the preferred width of images) and use it to edit your blog. It will be suitable and (hopefully) useful for.

  • A single-author blog for consistency
  • A (business) blog with multiple co-authors;
  • A business managing multiple paid contributors;
  • A business owner outsourcing content to ghost writers, etc.

Attention: This guide requires that your blog contributor has a wp-admin (Contributor) access to your blog. Otherwise, you may ask to send you the articles in HTML.

*Make a free copy of the checklist here. Read some explanations below*

Blog Post Checklist: Check All Prior to Hitting "Publish"

Structuring Standards

We know that breaking the article into sections is very important: blog readers tend to scan through the articles jumping from a subheading to a subheading (and probably reading the sections that seem most relevant to them).

One of the most efficient ways to capture your readers’ attention is to provide catchy subheadings that would summarize your article content nicely. Ideally, the reader of the article should understand what it is about by just scanning through the subheadings which should effectively summarize the article content while still encouraging the reader to go more indepth.

Subheadings

Other important guidelines that (may) go in this section:

  • Make your sentences and paragraphs short (that makes the whole article easier to read)
  • Introduce your article effectively in the opening paragraphs and encourage a discussion in the conclusion (by asking some questions)

Adding Links

Links are always great (unless links are brutally self-serving and anchor-text-dirty). I always encourage all my authors include links to any app, business or person they are mentioning. Links are user-friendly, so in my guidelines I always prompt contributors to:

  • Link to relevant articles on my blog;
  • Link to others relevant articles elsewhere (as well as sources of information, apps, etc)
  • Normally link words (not image files or subheadings)
  • Always make sure they are using the full URL and the original “clean” link (now a shortened version, stripping all tracking parameters, etc).

Lists

Including lists is also highly appreciated. A well-formatted list is likely to draw readers’ attention and make them stay.

Sadly, contributors tend to format lists in all possible ways using -, *, or any other inappropriate symbols instead proper <ul><li> coding. Therefore it’s a good idea to remind them of proper formatting here.

Images

Another very important element of any blog post. Images will never be added unless you clearly ask for them.

Here I specify the proper style and size of the images as well as encourage authors to include a relevant, catchy and properly attributed Creative Commons image.

Advertisement

Here are more checklists you may find useful for blogging:

FAQ: Blog Post Checklist: Check All Prior to Hitting “Publish”

What is the purpose of the blog post checklist?

The blog post checklist is designed to help bloggers and editors publish well-formatted articles consistently, without having to compromise on the creativity of the writing process.

Who can benefit from using this checklist?

The checklist can be beneficial for a range of people including single-author blogs, multi-author business blogs, business owners outsourcing content to ghost writers, and businesses managing multiple paid contributors.

What are the key components of the checklist?

Advertisement

The blog post checklist focuses on:

  • Structuring standards (including catchy subheadings and succinct paragraphs)
  • Proper linking to relevant articles and sources
  • Creating well-formatted lists
  • Adding appropriately styled and attributed images.

How should the article checklist be crafted?

The checklist should be as short, concise, and easy to understand as possible, with important points highlighted, possibly with the use of screenshots.

How can readers access the checklist?

Make a free copy of the checklist from the link and modify it according to their preferences and needs.

If you find the guidelines useful or have any improvements to suggest, please comment below!

Advertisement
The following two tabs change content below.
Blog Post Checklist Check All Prior to Hitting Publish

I am Ann Smarty, owner of SEOsmarty.com. I’ve been in the SEO industry for two decades. I am the former Editor-in-Chief of Search Engine Journal and a contributor to Mashable. These days I am running Viral Content Bee and writing for Moz, Buzzsumo, Wix and many others!

Blog Post Checklist Check All Prior to Hitting Publish
Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

How To Cultivate Focus And Execute Better

Published

on

By

Kevin Indig's Growth Memo for SEJ

Focus is mission-critical, but most companies don’t lower the gravitational forces pulling on attention.

As a result, large companies with too much mass have a hard time navigating and adapting to the quickly changing organic growth landscape:

No platform has as many changes of requirements. Over the last 3 years, Google launched 8 Core, 19 major and 75-150 minor updates. The company mentions thousands of improvements every year.

As individuals, we live in a distracted world, where one of the most important skills is managing attention. How did we think teams are any different?

Image Credit: Lyna ™

Boost your skills with Growth Memo’s weekly expert insights. Subscribe for free!

No Win Without Focus

Sir Isaac Newton realized that the sun’s gravity causes planets to orbit it in an elliptical path. Gravity in the workplace is a distraction from the focus of individuals and teams:

Advertisement
  • Meetings.
  • Fire drills.
  • Red tape.
  • Strategy pivots.
  • Too many goals.
  • Alignment overhead.
  • Too many cooks in the kitchen.
  • Procurement and legal rabbit holes.
  • Non-critical emails and Slack messages.

The larger a company gets, the stronger its gravity.

Taken to an extreme, it takes companies forever to launch even a single feature, and they fall behind the competition.

For individuals, gravity is even more consequential: Scattered attention means getting nothing done, having no impact, and likely being fired. Worse, people get exhausted and burned out in the process.

“Tranquility comes from doing less,” (Ryan Holiday), but a lot of teams execute scatter-brained like a teenager multitasking between Netflix, TikTok, and texting.

Individual and team focus are connected at the hip. When a team is distracted, it transfers to individuals. Two-thirds of people struggle to find the energy to do their job.

Whenever I get overwhelmed, my brain tells me to open my email inbox and look for a quick dopamine hit. But finding quick tasks and busy work is no achievement.

Real impact comes from working through tedious, complex problems.

Advertisement

We cannot erase gravity, but we can do five things better:

  1. Communication.
  2. Prioritization.
  3. Strategy.
  4. Red Tape.
  5. Meetings.

Better Communication

Unclear communication is one of the biggest attention drainers. We waste a lot of time deciphering what other people mean.

At Shopify, we had a very high bar for what internal communications went out to the Growth org and how they would be framed.

It’s easy to @ your whole team on Slack, but what people really need is key information:

  • What’s going on?
  • How is it relevant to me?
  • What do I need to know/do?

Lazy communication has massive speed cost. In the book “Smart Brevity,” the authors provide a simple framework for writing clear statements:

  1. Start with a muscular tease that grabs attention with six or fewer strong words.
  2. Explain what recipients need to know in the first sentence.
  3. Explain why it matters.
  4. Offer a choice to go deeper by providing more optional context.

Most important: Think about one thing you want people to remember – not more. Nobody has time to read a Slack novel.

Better Prioritization

At PayPal, Peter Thiel established a culture of hardcore focus. He would only discuss their No. 1 priority with managers and hold them accountable for just their one main contribution to the company.

Focus is a forcing function to separate the wheat from the chaff when it comes to what to spend time on.

No one could be a better example of hardcore prioritization than engineers. If you want to get something on the ENG roadmap, something else has to give.

Advertisement

An effective roadmap operating system has only a few lines of code:

  • What’s the goal?
  • What are the top 3 things that get us there?
  • For those three things, what do we need in terms of people, assets, time, support from other teams, and tools?
  • For those three things, who does what, by when?
  • Defend yourself and your team as much as possible from anything else.

You never look back at your time at a company and say, “Man, my fourth, fifth, and sixth priority back then really hit home,” but you might remember the impact of priorities one, two, and three.

Cal Newport’s new book, “Slow Productivity,” mentions doing less as one of the top ways to do better work.

But the advice I like the most is doubling the time you think a project takes.

Doubling automatically trims your roadmap by probably 50% but makes it more likely that you deliver on time and deliver well.

A big part of moving ourselves into an overcommitment corner is underestimating how long projects take (I think I wrote the last sentence at least as much for myself as for you).

Better Strategies

Poor strategies are hard to follow and confuse the team.

Advertisement

In my experience, managers want to get fancy, but they miss the most important point: A good strategy means doing something different from the competition.

Instead of outworking contenders, you want to do something that’s unique and leans to your competitive advantage.

Pairing differentiation with prioritization, your three most important projects should underline how you achieve the same goal in a different way as your competitors.

For example, instead of writing 100 blog articles, can you build a programmatic play or a list of tools? Or can you leverage contributors who write the content instead of a large in-house team?

I also found that most strategies simply aren’t clear. A simple test for clarity is to up or downsize the surface you have to explain it: Can you express your strategy in one paragraph (TL;DR), one page (high-level), and one doc (in-depth)?

Less Red Tape

Red tape in the form of excessive bureaucracy kills execution. I’ve seen many companies that take many weeks and endless alignment meetings before being able to sign up for a simple SaaS tool.

Advertisement

Procurement and legal teams can slow companies down and frustrate teams beyond means.

The key to having speed and a good evaluation process is clear guidelines when legal or procurement steps in.

With one of my former clients, the fastest-growing fintech startup in history, we sat down with the legal team and got a full download on guardrails. What can we say and what not? When do we have to get legal approval, and when can we move forward without it?

This is a task for the team manager or org leader. While tedious, the good news is that once the borders have been established, teams can move forward faster and focus on execution.

Fewer Meetings

Tobi Lütke, founder and CEO of Shopify, called meetings a “bug.” The leadership team regularly deployed the “Chaos Monkey,” a script that deletes all recurring meetings with more than two participants. Other companies set guardrails around time. Calendly restricts meetings to noon until 5 p.m.

Most meetings are poorly run, unnecessary, or simply a way for people to socialize.

Advertisement

Besides an agenda, every meeting should have a clear purpose. There really are only three types of meetings: socializing, information sharing, and decision-making.

Building relationships in the workplace is important, and there is nothing wrong with socializing. It’s important to be explicit and avoid meeting to “talk about Project X” while really wanting to socialize.

Information-sharing meetings are best done async. Instead of getting a large group of people together, record your message in a video or write a memo.

Decision-making meetings should be led by the decision maker and come with a pre-read.

The problem with many large organizations is that decisions are poorly framed; it’s unclear who makes the decision, and the decision-maker doesn’t have explicit criteria for how to make the decision.

Outlook: Can AI Help Us Regain Focus?

Show me how focused your team is, and I’ll show you a team that will win.

Advertisement

High gravity in large organizations, on the other hand, is an ask to be disrupted by a smaller, more agile player. The good news is that technology is working against gravity – at least in the workplace.

AI has the potential to help us find fragmented information, force clarity, and take over bland admin tasks that drain time so we can focus on things that matter.

Microsoft’s Future of Work report concludes:

“Organizational knowledge is fragmented across documents, conversations, apps and devices, but LLMs hold the potential to gather and synthesize this information in ways that were previously impossible”.

In the future, we’ll be able to ask LLMs questions about any internal process, like “What are our top goals?” or “Does this need legal review?” The freed-up time allows us to refine our strategies and get work done.

That future still seems a few years away. Until then, we can do a lot to improve our attention.


Layoffs.fyi

Advertisement

Marketing spending shows signs of growth, but AI adoption is slow: report; Beyond belt-tightening: How marketing can drive resiliency during uncertain times

Will AI Fix Work?

Workers Now Spend Two Full Days a Week on Email and in Meetings

Microsoft New Future of Work Report 2023


Featured Image: Paulo Bobita/Search Engine Journal

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Apple’s “Intelligent Search” Will Summarize Webpages

Published

on

By

Apple Intelligent Search

A report based on independently verified accounts notes that Apple’s Safari 18 will come with an Intelligent Search feature that summarize webpages in response to search queries and there may be a new feature called Web Eraser that allows users to permanently remove text, images, advertisements from webpages.

The Potential For Disruption By Apple AI Search

Apple has been gathering website data for years through its Internet crawler that uses the user agent, Applebot. The harvested data has in the past been used in the context of Siri and Apple’s Spotlight Suggestions feature.

Many in the search community have been aware of Applebot and have welcomed the prospect of a new search engine from Apple but despite constant crawling Apple has not released a search engine. A reason Apple has not released a standalone search engine may be that it’s become apparent that the best way to challenge Google Search is with a technology that replaces search engines altogether, much like how the Apple iPhone made digital cameras obsolete.

The latest news coming out about Safari 18 appears to confirm that supplanting Google is the strategy that Apple is pursuing.

The approach Apple is taking has the potential to disrupt not just search engines but also the search optimization and publishing industry, who have have been waiting years for an Apple search engine. But the extent of that disruption depends on how Apple implements their AI web search summarizer.

Advertisement

Webpage Summarization

Although news reports didn’t provide details as to how the new search result summaries will appear, it seems reasonable to speculate that Apple will provide attribution in the form of a link to websites so that users can click through to the website.

This is what was reported:

“With the release of Safari 18, Apple is expected to introduce article summarization through a new Intelligent Search feature — meaning users will have the option to generate a brief summary of any webpage currently on screen.

Apple’s built-in AI software can analyze the keywords and phrases within a webpage or document and generate a short summary containing only the most important information.”

SEOs have been giddy about the prospect of an Apple search engine for years. It now appears that the Google Killer they’ve been waiting for could possibly result in less traffic from search queries but to what extent it’s impossible to tell at this point.

One search marketing expert mused in a private chat that if Intelligent Search summarizes more than it links out then that may signal it’s time to start selling off the domain names they’ve invested in.

On-Device Processing

An interesting feature of the text summarization is that the technology that creates the summary (called Ajax) resides on the mobile device itself. What Ajax does is extract keywords, entities, and use the data to identify the topic and a loose summary of a webpage which is then turned into a text summary for the user.

This is how the functionality is described:

Advertisement

“In analyzing texts, the software takes into account all relevant information available. It can recognize and classify entities such as companies, people, and locations. For instance, if a name appears at the top of a text, the software will likely recognize the name as belonging to the text’s author.”

Apple Also Plans A Web Eraser

As if an Apple search summarizer isn’t bad enough Apple reportedly has a “Web Eraser” functionality planned for Safari. Web Eraser is a feature that removes content from webpages so that site visitors don’t have to look at it anymore. Things like advertising, videos, comments, suggested reading and maybe even popups could be permanently blocked by the Web Eraser.  Once a user “erases” a block of content from a webpage that block stays erased for the site visitor on subsequent visits.

According to the a report about the Apple Web Eraser:

“The feature is expected to build upon existing privacy features within Safari and will allow users to erase unwanted content from any webpage of their choosing. Users will have the option to erase banner ads, images, text or even entire page sections, all with relative ease.”

Technological Disruptions

It’s a natural response to experience anxiety in the face of changes. For many, the dawning of AI Search is their first experience of a major change. But for those of us who have been in search for 25+ years we have experienced and grown accustomed to sudden and transformative changes that alter publishing and SEO. I tend to feel that Apple’s implementation of an AI search engine that summarizes websites will be disruptive but not to the point that it harms websites. It’s in Apple’s self-interest to not disrupt the Internet to the point of extinction.

Featured Image by Shutterstock/frantic00

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS