Connect with us

SEO

Bing Revamps Crawl System To Enhance Efficiency

Published

on

Bing Revamps Crawl System To Enhance Efficiency

According to a recent study by Bing, most websites have XML sitemaps, with the “lastmod” tag being the most critical component of these sitemaps.

The “lastmod” tag indicates the last time the webpages linked by the sitemap were modified and is used by search engines to determine how often to crawl a site and which pages to index.

However, the study also revealed that a significant number of “lastmod” values in XML sitemaps were set incorrectly, with the most prevalent issue being identical dates on all sitemaps.

Upon consulting with web admins, Microsoft discovered that the dates were set to the date of sitemap generation rather than content modification.

To address this issue, Bing is revamping its crawl scheduling stack to better utilize the information provided by the “lastmod” tag in sitemaps.

This will improve crawl efficiency by reducing unnecessary crawling of unchanged content and prioritizing recently updated content.

The improvements have already begun on a limited scale and are expected to roll out by June fully.

Additionally, Microsoft has updated sitemap.org for improved clarity by adding the following line:

“Note that the date must be set to the date the linked page was last modified, not when the sitemap is generated.”

How To Use The Lastmod Tag Correctly

To correctly set the “lastmod” tag in a sitemap, you should include it in the <url> tag for each page in the sitemap.

The date should be in W3C Datetime format, with the most commonly used formats being YYYY-MM-DD or YYYY-MM-DDThh:mm:ssTZD.

The date should reflect the last time the page was modified and should be updated regularly to ensure that search engines understand the relevance and frequency of updates.

Here’s an example code snippet:

<?xml version=”1.0″ encoding=”UTF-8″?>

<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>

   <url>

      <loc>http://www.example.com/</loc>

      <lastmod>2023-01-23</lastmod>      

   </url>

Google’s Advice: Use Lastmod Tag After Significant Changes Only

Google’s crawlers also utilize the “lastmod” tag, and the suggestions on using it by both major search engines are similar.

Google Search Advocate John Mueller recently discussed the lastmod tag in the January edition of Google’s office-hours Q&A sessions.

It’s worth noting that Google recommends only using the “lastmod” tag for substantial modifications, which was not mentioned in Microsoft’s blog post.

Changing the date in the lastmod tag after minor edits can be viewed as an attempt to manipulate search snippets.

In Summary

Microsoft’s recent study and efforts to improve the utilization of the “lastmod” tag in sitemaps will result in more efficient and effective webpage crawling.

Publishers are encouraged to regularly update their sitemaps and lastmod tags to ensure that their pages are correctly indexed and easily accessible by search engines.


Featured Image: mundissima/Shutterstock

Source: Microsoft



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

5 Questions Answered About The OpenAI Search Engine

Published

on

By

5 Questions Answered About The OpenAI Search Engine

It was reported that OpenAI is working on a search engine that would directly challenge Google. But details missing from the report raise questions about whether OpenAI is creating a standalone search engine or if there’s another reason for the announcement.

OpenAI Web Search Report

The report published on The Information relates that OpenAI is developing a Web Search product that will directly compete with Google. A key detail of the report is that it will be partly powered by Bing, Microsoft’s search engine. Apart from that there are no other details, including whether it will be a standalone search engine or be integrated within ChatGPT.

All reports note that it will be a direct challenge to Google so let’s start there.

1. Is OpenAI Mounting A Challenge To Google?

OpenAI is said to be using Bing search as part of the rumored search engine, a combination of a GPT-4 with Bing Search, plus something in the middle to coordinate between the two .

In that scenario, what OpenAI is not doing is developing its own search indexing technology, it’s using Bing.

What’s left then for OpenAI to do in order to create a search engine is to devise how the search interface interacts with GPT-4 and Bing.

And that’s a problem that Bing has already solved by using what it Microsoft calls an orchestration layer. Bing Chat uses retrieval-augmented generation (RAG) to improve answers by adding web search data to use as context for the answers that GPT-4 creates. For more information on how orchestration and RAG works watch the keynote at Microsoft Build 2023 event by Kevin Scott, Chief Technology Officer at Microsoft, at the 31:45 minute mark here).

If OpenAI is creating a challenge to Google Search, what exactly is left for OpenAI to do that Microsoft isn’t already doing with Bing Chat? Bing is an experienced and mature search technology, an expertise that OpenAI does not have.

Is OpenAI challenging Google? A more plausible answer is that Bing is challenging Google through OpenAI as a proxy.

2. Does OpenAI Have The Momentum To Challenge Google?

ChatGPT is the fastest growing app of all time, currently with about 180 million users, achieving in two months what took years for Facebook and Twitter.

Yet despite that head start Google’s lead is a steep hill for OpenAI to climb.  Consider that Google has approximately 3 to 4 billion users worldwide, absolutely dwarfing OpenAI’s 180 million.

Assuming that all 180 million OpenAI users performed an average of 4 searches per day, the daily number of searches could reach 720 million searches per day.

Statista estimates that there are 6.3 million searches on Google per minute which equals over 9 billion searches per day.

If OpenAI is to compete they’re going to have to offer a useful product with a compelling reason to use it. For example, Google and Apple have a captive audience on mobile device ecosystem that embeds them into the daily lives of their users, both at work and at home. It’s fairly apparent that it’s not enough to create a search engine to compete.

Realistically, how can OpenAI achieve that level of ubiquity and usefulness?

OpenAI is facing an uphill battle against not just Google but Microsoft and Apple, too. If we count Internet of Things apps and appliances then add Amazon to that list of competitors that already have a presence in billions of users daily lives.

OpenAI does not have the momentum to launch a search engine to compete against Google because it doesn’t have the ecosystem to support integration into users lives.

3. OpenAI Lacks Information Retrieval Expertise

Search is formally referred to as Information Retrieval (IR) in research papers and patents. No amount of searching in the Arxiv.org repository of research papers will surface papers authored by OpenAI researchers related to information retrieval. The same can be said for searching for information retrieval (IR) related patents. OpenAI’s list of research papers also lacks IR related studies.

It’s not that OpenAI is being secretive. OpenAI has a long history of publishing research papers about the technologies they’re developing. The research into IR does not exist. So if OpenAI is indeed planning on launching a challenge to Google, where is the smoke from that fire?

It’s a fair guess that search is not something OpenAI is developing right now. There are no signs that it is even flirting with building a search engine, there’s nothing there.

4. Is The OpenAI Search Engine A Microsoft Project?

There is substantial evidence that Microsoft is furiously researching how to use LLMs as a part of a search engine.

All of the following research papers are classified as belonging to the fields of Information Retrieval (aka search), Artificial Intelligence, and Natural Language Computing.

Here are few research papers just from 2024:

Enhancing human annotation: Leveraging large language models and efficient batch processing
This is about using AI for classifying search queries.

Structured Entity Extraction Using Large Language Models
This research paper discovers a way to extracting structured information from unstructured text (like webpages). It’s like turning a webpage (unstructured data) into a machine understandable format (structured data).

Improving Text Embeddings with Large Language Models (PDF version here)
This research paper discusses a way to get high-quality text embeddings that can be used for information retrieval (IR). Text embeddings is a reference to creating a representation of text in a way that can be used by algorithms to understand the semantic meanings and relationships between the words.

The above research paper explains the use:

“Text embeddings are vector representations of natural language that encode its semantic information. They are widely used in various natural language processing (NLP) tasks, such as information retrieval (IR), question answering…etc. In the field of IR, the first-stage retrieval often relies on text embeddings to efficiently recall a small set of candidate documents from a large-scale corpus using approximate nearest neighbor search techniques.”

There’s more research by Microsoft that relates to search, but these are the ones that are specifically related to search together with large language models (like GPT-4.5).

Following the trail of breadcrumbs leads directly to Microsoft as the technology powering any search engine that OpenAI is supposed to be planning… if that rumor is true.

5. Is Rumor Meant To Steal Spotlight From Gemini?

The rumor that OpenAI is launching a competing search engine was published on February 14th. The next day on February 15th Google announced the launch of Gemini 1.5, after announcing Gemini Advanced on February 8th.

Is it a coincidence that OpenAI’s announcement completely overshadowed the Gemini announcement the next day? The timing is incredible.

At this point the OpenAI search engine is just a rumor.

Featured Image by Shutterstock/rafapress

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Warning: Unpopular SEO writing opinion

Published

on

Warning: Unpopular SEO writing opinion

Unpopular opinion alert: Adding new blog posts may not help your site.

(No matter what that content marketing company told you.) 🙄

So many of my new clients — especially subject matter experts — don’t need new content (immediately).

They HAVE content — scads of it scattered across various platforms.

(Maybe that sounds familiar.)

What they DO need is someone to review their content and customer persona, pinpoint opportunities, and develop a baby-step approach to leveraging those older content assets.

Because there are always opportunities. 🔥

Before writing another word, ask…

  • Are you repurposing the content you have? Or are you writing it once and forgetting about it (which is so common)?
  • Is your customer/reader persona still accurate, or has your target audience changed post-COVID?
  • Do your sales pages showcase your benefits and speak to your customers’ pain points? Or are they flat and dull?
  • Does your content sound like YOU with a point of view? Or is there a massive disconnect between how you talk to clients and the words you use on your site?
  • When did you last take a peek at your old sales emails and email welcome sequences? Could updating those assets make you more money?
  • Isn’t it time to save time (and budget) and leverage your existing content?

If you need help untangling your content and messaging, let me know. I love creating content order out of chaos.

After all…

 

Warning Unpopular SEO writing opinion

 

What do you think? Leave your comment below.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Bans Impersonation In Ads

Published

on

By

Google Bans Impersonation In Ads

Google bans impersonation and false affiliation in ads, enforcing policy changes in March.

  • Google bans impersonation and false affiliation in ads.
  • Policy enforcement starts in March.
  • Violators will be banned from Google Ads.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS