Connect with us

NEWS

Data Seemingly Proves Googlebot Crawling Has Slowed via @sejournal, @martinibuster

Published

on

There have been an ongoing discussions over the past few weeks across social media that Googlebot has dramatically reduced its crawling. For example, the founder of a web crawl analysis service tweeted a graph showing how Google’s crawl activity has declined since November 11, 2021.

Although the indexing slowdown doesn’t affect all sites, many on Twitter and Reddit agree that something changed at Google with respect to indexing and prove it with screenshots of Googlebot activity.

Evidence of Reduced Crawling

Anecdotal evidence of Google crawling anomalies have been stacking up on social media. The problem with social media is that one can literally make any observation about Google and nearly be guaranteed to receive agreement.

Anecdotal is interesting but not as good as data backed observations, which is what appeared recently on Twitter.

A founder of crawler and log analysis service Seolyzer (@Seolyzer_io) posted a graph of Google crawling behavior that showed a dramatic drop off of crawling activity beginning on November 11th.

Advertisement

Continue Reading Below

He posted:

“Googlebot is on strike! Googlebot has drastically reduced its crawl activity on many large sites since November 11 at 6PM (GMT).”

304 Server Response Code and Googlebot Crawling

Some have noted a pattern with Googlebot suddenly no longer crawling pages that serve a 304 server response code.

A 304 response code means 304 (Not Modified).

That response code is generated by a server when a browser (or Googlebot) makes a conditional request for a page.

That means that a browser (or Googlebot) tells the server it has a web page saved in cache so don’t bother serving it unless that page has been updated (modified).

Here is a definition of the 304 (Not Modified) server response code from the HTTP Working Group:

“The 304 (Not Modified) status code indicates that a conditional GET or HEAD request has been received and would have resulted in a 200 (OK) response if it were not for the fact that the condition evaluated to false.

In other words, there is no need for the server to transfer a representation of the target resource because the request indicates that the client, which made the request conditional, already has a valid representation; the server is therefore redirecting the client to make use of that stored representation as if it were the payload of a 200 (OK) response.”

Advertisement

Continue Reading Below

304 Response Causes Less Googlebot Crawling?

One person tweeted confirmation (in French) that on several sites with AMP that he monitors experienced a drop on pages that responded with a 304 response.

The person who posted the original tweet responded with a post of a graph showing how Google nearly stopped crawling pages that responded with a 304 server response code:

Others noticed a similar issue where pages serving a 304 response had drastically lower crawl rates:

Another person noticed reduced crawls on travel pages but a crawl increase on ecommerce pages:

Many others are sharing analytics and search console screenshots:

More data:

304 Response Code Should Not Alter Crawling

Google’s official developer help page documentation on Googlebot crawling states that a 304 response code should not impact crawling.

Here’s what Google’s official documentation advises:

“Googlebot signals the indexing pipeline that the content is the same as last time it was crawled.

The indexing pipeline may recalculate signals for the URL, but otherwise the status code has no effect on indexing.”

Advertisement

Continue Reading Below

Is it possible that Google has changed (permanently or temporarily) and that developer page is outdated?

Cookie Consent Theory

The 304 Server Response theory is one of many theories and solutions to explain why Googlebot might not index a web page.

One person tweeted that Google increased indexing after removing a cookie consent bar.

Why would a cookie response bar cause Google to not index a web page? Could the cookie consent bar have triggered a 304 response, causing Google to not index the page?

Reduced Googlebot Crawls Discussed at Reddit

The phenomenon of reduced Googlebot crawls were also discussed on Reddit.

A Redditor described how in the past articles from their successful site were indexed within 10 minutes of submitting them via Google Search Console.

They related that recently only half of new articles were being indexed.

Advertisement

Continue Reading Below

But that changed in November according to this Reddit post:

“For whatever reason now less than half of our new articles are indexing, even with me manually submitting them all right after publishing.”

Other redditors shared similar experiences:

“A lot of people are experiencing similar right now… Something seems to be going on with Google.”

“Something is up with Google indexing new posts…”

“My website is 17 years old… suddenly, the latest article took weeks to get indexed.”

Google Says Nothing is Broken

Google’s John Mueller responded to the questions on Reddit:

“I don’t see anything broken in the way Google indexes stuff at the moment. I do see us being critical about what we pick up for indexing though, as any search engine should.”

Is Google Testing New Crawling Patterns?

Bing in October announced an open source indexing protocol called IndexNow whose goal is to reduce how often crawlers crawl web pages in order to reduce how much energy is used at data centers for crawling and at servers for serving web pages. The new protocol benefits publishers because it speeds up the process of notifying search engines when pages are updated or created, resulting in faster indexing of quality web pages.

Advertisement

Continue Reading Below

In November Google announced that it would test the new IndexNow indexing protocol to see if there are benefits to it.

Saving energy and reducing the carbon footprint is one of our most important issues of today. Could it be that Google is improving on ways to reduce crawling without radically changing to a new protocol?

Has Google Reduced Web Page Crawling?

There are some claims that Google has stopped indexing altogether but that is incorrect. However there is significant discussion on social media backed with data to support that Googlebot indexing patterns have changed.

Searchenginejournal.com

NEWS

OpenAI Introduces ChatGPT Plus with Monthly Subscription of $20

Published

on

Open AI - Chat GPT

OpenAI, the leading artificial intelligence research laboratory, has launched a new product – ChatGPT Plus. The new product is an advanced version of its previous language model, ChatGPT, and is available for a monthly subscription of $20. The company aims to provide a more sophisticated and efficient conversational AI tool to its users through this new product.

ChatGPT Plus is a state-of-the-art language model that uses advanced deep learning algorithms to generate human-like responses to text inputs. The model has been trained on a massive corpus of text data, allowing it to generate coherent and contextually relevant responses. The model is designed to handle a wide range of conversational topics and can be integrated into various applications, such as chatbots, customer support systems, and virtual assistants.

One of the main advantages of ChatGPT Plus over its predecessor, ChatGPT, is its ability to generate responses in a more human-like manner. The model has been fine-tuned to incorporate more advanced language processing techniques, which enable it to better understand the context and tone of a conversation. This makes it possible for the model to generate more nuanced and appropriate responses, which can greatly improve the user experience.

In addition to its advanced language processing capabilities, ChatGPT Plus also offers improved performance in terms of response generation speed and efficiency. The model has been optimized to run on faster hardware and has been fine-tuned to generate responses more quickly. This makes it possible for the model to handle a larger volume of requests, making it an ideal solution for businesses with high traffic websites or customer support centers.

The monthly subscription fee of $20 for ChatGPT Plus makes it an affordable solution for businesses of all sizes. The company has designed the pricing model in such a way that it is accessible to businesses of all sizes, regardless of their budget. This makes it possible for small businesses to take advantage of advanced conversational AI technology, which can greatly improve their customer engagement and support.

OpenAI has also made it easy to integrate ChatGPT Plus into various applications. The company has provided a comprehensive API that allows developers to easily integrate the model into their applications. The API supports a wide range of programming languages, making it possible for developers to use the technology regardless of their preferred programming language. This makes it possible for businesses to quickly and easily incorporate conversational AI into their operations.

In conclusion, OpenAI’s launch of ChatGPT Plus is a significant development in the field of conversational AI. The new product offers advanced language processing capabilities and improved performance, making it an ideal solution for businesses of all sizes. The affordable pricing model and easy integration make it accessible to businesses of all sizes, and the advanced language processing capabilities make it possible for businesses to improve their customer engagement and support. OpenAI’s ChatGPT Plus is set to revolutionize the conversational AI industry and bring advanced technology within the reach of businesses of all sizes.

Visit OpenAI.com to read more and to get the latest news about ChatGPT.

Continue Reading

NEWS

What can ChatGPT do?

Published

on

ChatGPT Explained

ChatGPT is a large language model developed by OpenAI that is trained on a massive amount of text data. It is capable of generating human-like text and has been used in a variety of applications, such as chatbots, language translation, and text summarization.

One of the key features of ChatGPT is its ability to generate text that is similar to human writing. This is achieved through the use of a transformer architecture, which allows the model to understand the context and relationships between words in a sentence. The transformer architecture is a type of neural network that is designed to process sequential data, such as natural language.

Another important aspect of ChatGPT is its ability to generate text that is contextually relevant. This means that the model is able to understand the context of a conversation and generate responses that are appropriate to the conversation. This is accomplished by the use of a technique called “masked language modeling,” which allows the model to predict the next word in a sentence based on the context of the previous words.

One of the most popular applications of ChatGPT is in the creation of chatbots. Chatbots are computer programs that simulate human conversation and can be used in customer service, sales, and other applications. ChatGPT is particularly well-suited for this task because of its ability to generate human-like text and understand context.

Another application of ChatGPT is language translation. By training the model on a large amount of text data in multiple languages, it can be used to translate text from one language to another. The model is able to understand the meaning of the text and generate a translation that is grammatically correct and semantically equivalent.

In addition to chatbots and language translation, ChatGPT can also be used for text summarization. This is the process of taking a large amount of text and condensing it into a shorter, more concise version. ChatGPT is able to understand the main ideas of the text and generate a summary that captures the most important information.

Despite its many capabilities and applications, ChatGPT is not without its limitations. One of the main challenges with using language models like ChatGPT is the risk of generating text that is biased or offensive. This can occur when the model is trained on text data that contains biases or stereotypes. To address this, OpenAI has implemented a number of techniques to reduce bias in the training data and in the model itself.

In conclusion, ChatGPT is a powerful language model that is capable of generating human-like text and understanding context. It has a wide range of applications, including chatbots, language translation, and text summarization. While there are limitations to its use, ongoing research and development is aimed at improving the model’s performance and reducing the risk of bias.

** The above article has been written 100% by ChatGPT. This is an example of what can be done with AI. This was done to show the advanced text that can be written by an automated AI.

Continue Reading

NEWS

Google December Product Reviews Update Affects More Than English Language Sites? via @sejournal, @martinibuster

Published

on

Google’s Product Reviews update was announced to be rolling out to the English language. No mention was made as to if or when it would roll out to other languages. Mueller answered a question as to whether it is rolling out to other languages.

Google December 2021 Product Reviews Update

On December 1, 2021, Google announced on Twitter that a Product Review update would be rolling out that would focus on English language web pages.

The focus of the update was for improving the quality of reviews shown in Google search, specifically targeting review sites.

A Googler tweeted a description of the kinds of sites that would be targeted for demotion in the search rankings:

“Mainly relevant to sites that post articles reviewing products.

Think of sites like “best TVs under $200″.com.

Goal is to improve the quality and usefulness of reviews we show users.”

Advertisement

Continue Reading Below

Google also published a blog post with more guidance on the product review update that introduced two new best practices that Google’s algorithm would be looking for.

The first best practice was a requirement of evidence that a product was actually handled and reviewed.

The second best practice was to provide links to more than one place that a user could purchase the product.

The Twitter announcement stated that it was rolling out to English language websites. The blog post did not mention what languages it was rolling out to nor did the blog post specify that the product review update was limited to the English language.

Google’s Mueller Thinking About Product Reviews Update

Screenshot of Google's John Mueller trying to recall if December Product Review Update affects more than the English language

Screenshot of Google's John Mueller trying to recall if December Product Review Update affects more than the English language

Product Review Update Targets More Languages?

The person asking the question was rightly under the impression that the product review update only affected English language search results.

Advertisement

Continue Reading Below

But he asserted that he was seeing search volatility in the German language that appears to be related to Google’s December 2021 Product Review Update.

This is his question:

“I was seeing some movements in German search as well.

So I was wondering if there could also be an effect on websites in other languages by this product reviews update… because we had lots of movement and volatility in the last weeks.

…My question is, is it possible that the product reviews update affects other sites as well?”

John Mueller answered:

“I don’t know… like other languages?

My assumption was this was global and and across all languages.

But I don’t know what we announced in the blog post specifically.

But usually we try to push the engineering team to make a decision on that so that we can document it properly in the blog post.

I don’t know if that happened with the product reviews update. I don’t recall the complete blog post.

But it’s… from my point of view it seems like something that we could be doing in multiple languages and wouldn’t be tied to English.

And even if it were English initially, it feels like something that is relevant across the board, and we should try to find ways to roll that out to other languages over time as well.

So I’m not particularly surprised that you see changes in Germany.

But I also don’t know what we actually announced with regards to the locations and languages that are involved.”

Does Product Reviews Update Affect More Languages?

While the tweeted announcement specified that the product reviews update was limited to the English language the official blog post did not mention any such limitations.

Google’s John Mueller offered his opinion that the product reviews update is something that Google could do in multiple languages.

One must wonder if the tweet was meant to communicate that the update was rolling out first in English and subsequently to other languages.

It’s unclear if the product reviews update was rolled out globally to more languages. Hopefully Google will clarify this soon.

Citations

Google Blog Post About Product Reviews Update

Product reviews update and your site

Google’s New Product Reviews Guidelines

Write high quality product reviews

John Mueller Discusses If Product Reviews Update Is Global

Watch Mueller answer the question at the 14:00 Minute Mark

[embedded content]Searchenginejournal.com

Continue Reading

Trending

en_USEnglish