How 500 Error Codes Can Impact Google Indexing

Google’s John Mueller answered a question about at what point will 500 error response codes cause Googlebot to crawl content less often. Mueller explained scenarios where 500 response codes won’t affect crawling and when they may cause pages to drop from Google’s index.
Accidentally Blocked Googlebot and Caused 500 Errors
The person asking the question explained that their content delivery system was set up to block bad bots while allowing actual human users and legit bots to browse the site.
Everything was working fine until they updated their “server monitoring suite” and apparently the previous bot blocking filters and settings weren’t carried over to the new system.
Afterward they noticed that Google Search Console began listing 500 error response codes in response to Googlebot crawling.
The person finished explaining what happened then asked the question:
“The question came up from our technology team whether this represented real user impact and why we would look specifically at Googlebot and not real user metrics to prove that there’s an issue here.
So, given that context, I have a few questions.
The first is just to get the technology concern out …of the way.
From your perspective, how does Googlebot view 500 series errors and could you give any clarity on established …thresholds at which point Googlebot will …crawl source content …less, based on those errors?”
What is a 500 Error Code?
Servers communicate a variety of status response codes to browsers and bots. These response codes communicate, for example, whether the request for a page was successful, redirected to another page, blocked or unsuccessful.
- The 200 response code communicates that the request for a web page was successful.
- The 500 response code means that the request for a web page was unsuccessful.
The formal name of the 500 response code is:
"500 Internal Server Error"
There are other versions of the 500 response code that communicate a specific reason why the request for a web page was unsuccessful.
The 500 response code is somewhat generic in that it simply means that the page request was unsuccessful. But it often means that the server was down but not always.
Google Search Console and 500 Error Responses
It’s a good idea to monitor the Search Console for 500 Error Responses because they are a strong indicator that something is wrong.
Sometimes it could mean that the server is under heavy load, for example from too many bots and because of that it is unable to serve web pages.
In this case it appears that the problem was due to a misconfiguration in the Content Delivery Network (CDN) that inadvertently blocked Google from accessing web pages.
How Googlebot Indexing is Affected by 500 Errors
Mueller provided a helpful explanation of the steps Google takes in response to 500 error codes and how eventually continued errors could lead to those web pages being dropped from Google’s search index.
Google’s John Mueller responded:
“We don’t have any strong thresholds on that.
But essentially what happens with 500 errors is we’ll try to retry them.
And if we continue to see …the 500 errors then we will …slow down crawling.
And if we continue to see that there are 500 errors then we will drop those URLs from the index.
So that’s something if every now and then individual pages have a 500 error, it’s like no big deal.
We will retry them, they’ll remain indexed and the next time we retry them that’s fine.”
Next, Mueller explains that the reason Googlebot slows down the crawling is in case Googlebot itself is causing the server to overload.
“But if a large part of a site consistently has 500 errors and we might assume that maybe we’re causing the problem and we’ll slow down crawling of the whole site and at some point we’ll say well, it looks like these pages are really gone, we’re going to drop them.
So that’s essentially the effects that you would see there and if you’re talking about a large site and wondering like what percentage of 500 errors is okay… I don’t know.
My feeling is if you’re seeing something more than one percent then that sounds like something is kind of broken and probably would be something where we would start to slow down.
But I don’t think we have any hard thresholds where we’d say, this many requests and this many errors means this much slowing down.”
Monitor Search Console for 500 Errors
It’s up to every publisher to decide what their daily routine is. For some people checking the search console once a week is enough. Some may find it reassuring to check Search Console once a day in order to respond to unforeseen circumstances.
In any case, 500 error codes is something to monitor Search Console for. As John Mueller mentioned, large-scale 500 errors could be a flag that something is wrong and in need of immediate attention.
Citat
World Wide Web Consortium (W3C) List of Server Response Codes
How Does Google Respond to 500 Response Codes?
Watch the discussion at the 13:48 minute mark
Google Bard vs. ChatGPT: vilken är den bättre AI-chatboten?

Google Bard och ChatGPT are two of the most prominent artificial intelligence (AI) chatbots available in 2023. But which is better? Both offer natural language responses to natural language inputs, using machine learning and millions of data points to craft useful, informative responses. Most of the time. These AI tools aren’t perfect yet, but they point to an exciting future of AI assistant search and learning tools that will make information all the more readily available.
As similar as these chatbots are, they also have some distinct differences. Here’s how ChatGPT and Google Bard measure up against one another.
Which is better, Google Bard or ChatGPT?
This is a tricky question to answer, as at the time of writing, you can only use Google Bard if you’re part of a select group of early beta testers. As for its competition, you can use ChatGPT right now, completely for free. You may have to contend with a waitlist, but if you want to skip that, there’s a paid-for Plus version offering those interested in a more complete tool the option of paying for the privilege.
Still, when Google Bard becomes more widely available, it should offer credible competition for ChatGPT. Both use natural language models — Google Bard uses Google’s internal LaMDA (Language Model for Dialogue Applications), whereas ChatGPT uses an older GPT-3 language model. Google Bard bases its responses to questions on more recent data, with ChatGPT mainly trained on data that was available prior to 2021. This is similar to how Microsoft’s Bing Chat works.
We’ll have to reserve judgment on which is the more capable AI chatbot until we get time to play with Google Bard ourselves, but it looks set to be a close contest when it is more readily available.
Are Google Bard and ChatGPT available yet?
As mentioned, ChatGPT is available in free and paid-for tiers. You might have to sit in a queue for the free version for a while, but anyone can play around with its capabilities.
Google Bard is currently only available to limited beta testers and is not available to the wider public.

What’s the difference between Google Bard and ChatGPT?
ChatGPT and Google Bard are very similar natural language AI chatbots, but they have some differences, and are designed to be used in slightly different ways — at least for now. ChatGPT has been used for answering direct questions with direct answers, mostly correctly, but it’s caused a lot of consternation among white collar workers, like writers, SEO advisors, and copy editors, since it has also demonstrated an impressive ability to write creatively — even if it has faced a few problems with accuracy and plagiarism.
Still, Microsoft has integrated ChatGPT into its Bing search engine to give users the ability to ask direct questions of the search engine, rather than searching for terms of keywords to find the best results. It has also built it into its Teams communications tool, and it’s coming to the Edge browser in a limited form. The Opera browser has also pledged to integrate ChatGPT in the future.
ChatGPT | Google Bard |
Accessible through ChatGPT site. Only text responses are returned via queries. | Integrated with Google Search. You only need to change a Google setting to get your regular search results when using Google Bard AI, and vice versa. |
ChatGPT produces answers from its trained database from 2021 and before. | Google Apprentice Bard AI will be able to answer real-time questions. |
Based on GPT (Generative Pre-trained Transformer). | Based on LaMDA (Language Model for Dialogue Applications). |
Service has a free and paid plan option (called ChatGPT Plus). | Service is free. |
Has built-in plagiarism tool called GPT-2 Output Detector. | No built-in plagiarism detection tool. |
Available now | Still in beta test phase |
Google Bard was mainly designed around augmenting Google’s own search tool, however it is also destined to become an automated support tool for businesses without the funds to pay for human support teams. It will be offered to customers through a trained AI responder. It is likely to be integrated into the Chrome browser and its Chromium derivatives before long. Google is also expected to open up Google Bard to third-party developers in the future.
Under the hood, Google Bard uses Google’s LaMDA language model, while ChatGPT uses its own GPT3 model. ChatGPT is based on slightly older data, restricted in its current GPT3 model to data collected prior to 2022, while Google Bard is built on data provided on recent years too. However, that doesn’t necessarily make it more accurate, as Google Bard has faced problems with incorrect answers to questions, even in its initial unveiling.
ChatGPT also has a built-in plagiarism checker, while Google Bard does not, but Google Bard doesn’t have the creative applications of ChatGPT just yet.
Google to pay $391.5 million settlement over location tracking, state AGs say

Google has agreed to pay a $391.5 million settlement to 40 states to resolve accusations that it tracked people’s locations in violation of state laws, including snooping on consumers’ whereabouts even after they told the tech behemoth to bug off.
Louisiana Attorney General Jeff Landry said it is time for Big Tech to recognize state laws that limit data collection efforts.
“I have been ringing the alarm bell on big tech for years, and this is why,” Mr. Landry, a Republican, said in a statement Monday. “Citizens must be able to make informed decisions about what information they release to big tech.”
The attorneys general said the investigation resulted in the largest-ever multistate privacy settlement. Connecticut Attorney General William Tong, a Democrat, said Google’s penalty is a “historic win for consumers.”
“Location data is among the most sensitive and valuable personal information Google collects, and there are so many reasons why a consumer may opt out of tracking,” Mr. Tong said. “Our investigation found that Google continued to collect this personal information even after consumers told them not to. That is an unacceptable invasion of consumer privacy, and a violation of state law.”
Location tracking can help tech companies sell digital ads to marketers looking to connect with consumers within their vicinity. It’s another tool in a data-gathering toolkit that generates more than $200 billion in annual ad revenue for Google, accounting for most of the profits pouring into the coffers of its corporate parent, Alphabet, which has a market value of $1.2 trillion.
The settlement is part of a series of legal challenges to Big Tech in the U.S. and around the world, which include consumer protection and antitrust lawsuits.
Though Google, based in Mountain View, California, said it fixed the problems several years ago, the company’s critics remained skeptical. State attorneys general who also have tussled with Google have questioned whether the tech company will follow through on its commitments.
The states aren’t dialing back their scrutiny of Google’s empire.
Last month, Texas Attorney General Ken Paxton said he was filing a lawsuit over reports that Google unlawfully collected millions of Texans’ biometric data such as “voiceprints and records of face geometry.”
The states began investigating Google’s location tracking after The Associated Press reported in 2018 that Android devices and iPhones were storing location data despite the activation of privacy settings intended to prevent the company from following along.
Arizona Attorney General Mark Brnovich went after the company in May 2020. The state’s lawsuit charged that the company had defrauded its users by misleading them into believing they could keep their whereabouts private by turning off location tracking in the settings of their software.
Arizona settled its case with Google for $85 million last month. By then, attorneys general in several other states and the District of Columbia had pounced with their own lawsuits seeking to hold Google accountable.
Along with the hefty penalty, the state attorneys general said, Google must not hide key information about location tracking, must give users detailed information about the types of location tracking information Google collects, and must show additional information to people when users turn location-related account settings to “off.”
States will receive differing sums from the settlement. Mr. Landry’s office said Louisiana would receive more than $12.7 million, and Mr. Tong’s office said Connecticut would collect more than $6.5 million.
The financial penalty will not cripple Google’s business. The company raked in $69 billion in revenue for the third quarter of 2022, according to reports, yielding about $13.9 billion in profit.
Google downplayed its location-tracking tools Monday and said it changed the products at issue long ago.
“Consistent with improvements we’ve made in recent years, we have settled this investigation which was based on outdated product policies that we changed years ago,” Google spokesman Jose Castaneda said in a statement.
Google product managers Marlo McGriff and David Monsees defended their company’s Search and Maps products’ usage of location information.
“Location information lets us offer you a more helpful experience when you use our products,” the two men wrote on Google’s blog. “From Google Maps’ driving directions that show you how to avoid traffic to Google Search surfacing local restaurants and letting you know how busy they are, location information helps connect experiences across Google to what’s most relevant and useful.”
The blog post touted transparency tools and auto-delete controls that Google has developed in recent years and said the private browsing Incognito mode prevents Google Maps from saving an account’s search history.
Mr. McGriff and Mr. Monsees said Google would make changes to its products as part of the settlement. The changes include simplifying the process for deleting location data, updating the method to set up an account and revamping information hubs.
“We’ll provide a new control that allows users to easily turn off their Location History and Web & App Activity settings and delete their past data in one simple flow,” Mr. McGriff and Mr. Monsees wrote. “We’ll also continue deleting Location History data for users who have not recently contributed new Location History data to their account.”
• This article is based in part on wire service reports.
5 Tips to Boost Your Holiday Search Strategy

With the global economic downturn, inflation, ongoing supply chain challenges, and uncertainty due to the Ukraine war, this year’s holiday shopping season promises to be very challenging. Will people be in the mood to spend despite the gloom? Or will they rein in their enthusiasm and save for the year ahead?
With these issues in mind, here are five considerations to support your search engine optimization strategy this holiday shopping season:
1. Start early.
Rising prices are likely to mean shoppers will start researching their holiday spending earlier than ever to nab the best bargains. Therefore, retailers must roll out their holiday product and category pages — and launch any promotions — sooner to ensure their pages get crawled and indexed by search engines in good time.
Vissa e-handelsbutiker lyckas få sina sidor att rankas tidigt genom att uppdatera och återanvända samma del av webbplatsen för högtidsinnehåll och kampanjer, rotera mellan innehåll för jul, mors dag, alla hjärtans presenter, försäljning av fjärde juli, etc. Detta tillvägagångssätt kan hjälpa dig att behålla den fart, länkar och auktoritet du bygger upp med Google och få dina semestersidor synliga och rankas snabbt.
2. Gör forskning till en ännu större prioritet.
Med all osäkerhet i år är det viktigt att använda SEO-forskning för att identifiera trendiga säsongsbetonade nyckelord och sökfraser i din detaljhandelskategori – och sedan optimera innehållet därefter.
Med verktyg som Google Trender kan du extrahera användbara insikter baserat på vilka typer av sökningar människor gör. Till exempel, med många modeåterförsäljare som nu tar betalt för produktreturer, kommer prioritering av sökord som "gratis retur" att få mer sökkraft? Och när pengarna är snävare, kommer konsumenterna att hålla fast vid varumärken de litar på snarare än något nytt – vilket betyder att varumärkessökningar kan vara högre?
3. Använd Google Shopping mer.
För att få ut det mesta av sina semesterutgifter är det mer sannolikt att konsumenter vänder sig till onlinemarknadsplatser som Google Shopping eftersom de gör det lättare att jämföra produkter, funktioner och priser, samt att identifiera de bästa erbjudandena både online och i närliggande butiker .
Använd därför ett kombinerat tillvägagångssätt som inkluderar listning i Google Shopping och samtidigt optimera produktdetaljsidor på din e-handelssida för att säkerställa att de är unika och ger mer värde än konkurrenternas sidor. Var exakt med produktnamn på Google Shopping (innehåller namnen t.ex. de ord som folk söker efter?); se till att du tillhandahåller all nödvändig information som Google kräver; och sätt ett pris som inte är alltför långt från konkurrenterna.
4. Ge andra sökkällor den uppmärksamhet de förtjänar.
Tidigare i år Google själv erkänd att konsumenter – särskilt yngre konsumenter – börjar använda TikTok, Instagram och andra sociala medier för sökning. Faktiskt, forskning föreslår att 11 procent av produktsökningarna nu börjar på TikTok och 15 procent på Instagram. Särskilt yngre konsumenter är mer engagerade av visuellt innehåll, vilket kan förklara varför de anammar visuellt fokuserade sociala webbplatser för sökning. Så, som en del av din sökstrategi, skapa och dela innehåll på populära sociala medier som dina målkunder besöker.
På samma sätt, med människor som börjar sina shoppingsökningar på marknadsplatser som Amazon.com, bör optimering av alla listor du har på webbplatsen vara en del av din strategi. Och tack och lov, ju bättre optimerade dina produktdetaljsidor är för Amazon (med unikt, användbart innehåll), desto bättre kommer de att rankas på Google också!
5. Håll betald budget för sena tillfällen.
Den större osäkerheten och volatiliteten den här semesterperioden innebär att du måste hålla ett öga på kundernas beteende och vara redo att omfamna möjligheter som dyker upp senare. Att få höga organiska rankningar för sena kampanjer är alltid mer utmanande, så håll tillbaka en del av den betalda sökbudgeten för att hjälpa till att driva trafik till dessa sidor – till exempel via Google Ads. Viktiga sökord att inkludera i sensäsongens sökannonskampanjer inkluderar "leverans före jul" och "leverans samma dag". För lokalt inriktade sökannonser kan du överväga att "hämta när som helst före jul".
Utsikten till en tuff, oförutsägbar shoppingsäsong för helgerna innebär att sökteam måste lansera säsongsbetonade SEO-planer tidigt, noggrant spåra shoppares beteende och vara redo att anpassa sig när saker och ting förändras.
Marcus Pentzek är chefs SEO-konsult på Sökmetrics, den globala leverantören av sökdata, mjukvara och konsultlösningar.
-
SÖKMOTORER4 dagar sedan
Google uppdaterar policycenter för shoppingannonser och policycenter för gratisannonser
-
SEO4 dagar sedan
Hur man använder AI för att förbättra ditt SEO-innehållsskrivande [Webinar]
-
SÖKMOTORER5 dagar sedan
Google Local Service Ads skickar ut masspolicyöverträdelser
-
SÖKMOTORER5 dagar sedan
Google-sökning med mer detaljerade verktyg för biljämförelse
-
SEO5 dagar sedan
Googles sökrelationsteam undersöker Web3:s SEO-effekt
-
PPC5 dagar sedan
49 Fars dag Instagram bildtexter och färdiga bilder
-
WORDPRESS7 dagar sedan
Anpassade temadesigner blev precis enklare – WordPress.com Nyheter
-
SÖKMOTORER6 dagar sedan
Bing-videosökningsknapp "Mer så här".