Connect with us

SEO

500 Response on Robots.txt Fetch Can Impact Rich Results

Published

on

500 Response on Robots.txt Fetch Can Impact Rich Results

Google’s John Mueller received feedback about a bug in how Search Console validates rich results. Google will drop images from rich results because of an error in how a CDN that hosts the images handles a request for a non-existent robots.txt. The bug that was discovered was in how search console and Google’s rich results test will fail to alert the publisher of the error and subsequently give the structured data a successful validation.

A bug in the context of programming is when a software program behaves in an unexpected manner. A bug isn’t always a problem in the coding but as in this case, it could be a failure to anticipate an issue which in turn leads to unintended results, like this one.

The publisher asking the question tried to use Google’s tools to diagnose the reason why their rich results were disappearing and was surprised to find that they were of no use for this particular error.

While this issue was affecting the recipe rich results image preview in Google’s recipe rich results, this problem could also be an issue for other situations as well.

So it’s good to be aware of this problem as it might surface in other ways.

Advertisement

Recipe Rich Results Image Previews Disappeared

The person asking the question provided a background of what happened.

He related what happened:

“We ran into a bit of a tiger trap, I would say, in terms of rich recipe results.

We have hundreds of thousands of recipes which are indexed and there’s lots of traffic coming through from the recipe gallery.

And then… over a period of time it stopped.

And all of the meta data checked out and Google search Console was saying …this is all rich recipe content, it’s all good, it can be shown.

We finally noticed that in the preview, when you preview the result, the image was missing.

Advertisement

And it seems that there was a change at Google and that if a robots.txt was required in order for images to be retrieved, then nothing we could see in the tools was actually saying anything was invalid.

And so it’s a bit awkward right, when you check something to say “is this a valid rich recipe result?” and it says yea, it’s great, it’s absolutely great, we’ve got all the metadata.

And you check all the URLs and all the images are right, but it turns out behind the scenes, there was a new requirement that you have a robots.txt.”

John Mueller asked:

“How do you mean that you had to have a robots.txt?”

The person asking the question responded:

“What we found is, if you requested the robots.txt from our CDN, it gave you like a 500.

When we put a robots.txt there, immediately the previews started appearing correctly.

Advertisement

And that involves crawling and putting it onto a static site, I think.

So we operationally, we found adding that robots.txt did the job.”

John Mueller nodded his head and said:

“Yeah, okay.

So from our point of view, it’s not that a robots.txt file is required. But it has to have a proper result code.

So if you don’t have on, it should return 404.

If you do have one, then we can obviously read that.

Advertisement

But if you return a server error for the robots.txt file, then our systems will assume that maybe there is an issue with the server and we won’t crawl.

And that’s kind of something that’s been like that since the beginning.

But these kinds of issues where especially when you are on a CDN and it’s on a separate hostname, sometimes that’s really hard to spot.

And I imaging the rich results test, at least as far as I know, it focuses on the content that is on the HTML page.

So the JSON-LD markup that you have there, it probably doesn’t check to see if the images are actually fetchable.

And then if they can’t be fetched then, of course, we can’t use them in the carousel, too.

Advertisement

So that might be something that we need to figure out how to highlight better.”

500 Error Response for CDN Robots.txt Can Cause Issues

This is one of those show stopping SEO problems that are hard to diagnose but can cause a lot of negative issues as the person asking the question noted.

Normally a crawl for a robots.txt that is non-existent should result in a server response code of 404, which means that the robots.txt does not exist.

So if the request for a robots.txt file is generating a 500 response code then that’s an indication that something on the server or the CMS is misconfigured.

The short term solution is to upload a robots.txt file.

But it might be a good idea to dive into the CMS or server to check what the underlying issue is.

Advertisement

500 Response Code for a Robots.txt Fetch

The negative consequences for the recipes rich results preview because of a CDN that returns a 500 error response might be a rare issue.

A 500 server error response code sometimes happens when there is something unexpected or missing in the code and the server responds by ending the code processing and throwing the 500 response code.

For example, if you edit a PHP file and forget to indicate the end of a section of  code then that might cause the server to give up processing the code and throw a 500 response.

Whatever the reason for the error response when Google tried to fetch the robots.txt, this is a good issue to keep in mind for that rare situation when it happens to you.

Citation

CDN for Images and Recipe Rich Results Bug

Watch at the 51:45 Minute Mark

Advertisement




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

brightonSEO Live Blog

Published

on

brightonSEO Live Blog

Hello everyone. It’s April again, so I’m back in Brighton for another two days of sun, sea, and SEO!

Being the introvert I am, my idea of fun isn’t hanging around our booth all day explaining we’ve run out of t-shirts (seriously, you need to be fast if you want swag!). So I decided to do something useful and live-blog the event instead.

Follow below for talk takeaways and (very) mildly humorous commentary. 

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Further Postpones Third-Party Cookie Deprecation In Chrome

Published

on

By

Close-up of a document with a grid and a red stamp that reads "delayed" over the word "status" due to Chrome's deprecation of third-party cookies.

Google has again delayed its plan to phase out third-party cookies in the Chrome web browser. The latest postponement comes after ongoing challenges in reconciling feedback from industry stakeholders and regulators.

The announcement was made in Google and the UK’s Competition and Markets Authority (CMA) joint quarterly report on the Privacy Sandbox initiative, scheduled for release on April 26.

Chrome’s Third-Party Cookie Phaseout Pushed To 2025

Google states it “will not complete third-party cookie deprecation during the second half of Q4” this year as planned.

Instead, the tech giant aims to begin deprecating third-party cookies in Chrome “starting early next year,” assuming an agreement can be reached with the CMA and the UK’s Information Commissioner’s Office (ICO).

The statement reads:

Advertisement

“We recognize that there are ongoing challenges related to reconciling divergent feedback from the industry, regulators and developers, and will continue to engage closely with the entire ecosystem. It’s also critical that the CMA has sufficient time to review all evidence, including results from industry tests, which the CMA has asked market participants to provide by the end of June.”

Continued Engagement With Regulators

Google reiterated its commitment to “engaging closely with the CMA and ICO” throughout the process and hopes to conclude discussions this year.

This marks the third delay to Google’s plan to deprecate third-party cookies, initially aiming for a Q3 2023 phaseout before pushing it back to late 2024.

The postponements reflect the challenges in transitioning away from cross-site user tracking while balancing privacy and advertiser interests.

Transition Period & Impact

In January, Chrome began restricting third-party cookie access for 1% of users globally. This percentage was expected to gradually increase until 100% of users were covered by Q3 2024.

However, the latest delay gives websites and services more time to migrate away from third-party cookie dependencies through Google’s limited “deprecation trials” program.

The trials offer temporary cookie access extensions until December 27, 2024, for non-advertising use cases that can demonstrate direct user impact and functional breakage.

Advertisement

While easing the transition, the trials have strict eligibility rules. Advertising-related services are ineligible, and origins matching known ad-related domains are rejected.

Google states the program aims to address functional issues rather than relieve general data collection inconveniences.

Publisher & Advertiser Implications

The repeated delays highlight the potential disruption for digital publishers and advertisers relying on third-party cookie tracking.

Industry groups have raised concerns that restricting cross-site tracking could push websites toward more opaque privacy-invasive practices.

However, privacy advocates view the phaseout as crucial in preventing covert user profiling across the web.

With the latest postponement, all parties have more time to prepare for the eventual loss of third-party cookies and adopt Google’s proposed Privacy Sandbox APIs as replacements.

Advertisement

Featured Image: Novikov Aleksey/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

How To Write ChatGPT Prompts To Get The Best Results

Published

on

By

How To Write ChatGPT Prompts To Get The Best Results

ChatGPT is a game changer in the field of SEO. This powerful language model can generate human-like content, making it an invaluable tool for SEO professionals.

However, the prompts you provide largely determine the quality of the output.

To unlock the full potential of ChatGPT and create content that resonates with your audience and search engines, writing effective prompts is crucial.

In this comprehensive guide, we’ll explore the art of writing prompts for ChatGPT, covering everything from basic techniques to advanced strategies for layering prompts and generating high-quality, SEO-friendly content.

Writing Prompts For ChatGPT

What Is A ChatGPT Prompt?

A ChatGPT prompt is an instruction or discussion topic a user provides for the ChatGPT AI model to respond to.

Advertisement

The prompt can be a question, statement, or any other stimulus to spark creativity, reflection, or engagement.

Users can use the prompt to generate ideas, share their thoughts, or start a conversation.

ChatGPT prompts are designed to be open-ended and can be customized based on the user’s preferences and interests.

How To Write Prompts For ChatGPT

Start by giving ChatGPT a writing prompt, such as, “Write a short story about a person who discovers they have a superpower.”

ChatGPT will then generate a response based on your prompt. Depending on the prompt’s complexity and the level of detail you requested, the answer may be a few sentences or several paragraphs long.

Use the ChatGPT-generated response as a starting point for your writing. You can take the ideas and concepts presented in the answer and expand upon them, adding your own unique spin to the story.

Advertisement

If you want to generate additional ideas, try asking ChatGPT follow-up questions related to your original prompt.

For example, you could ask, “What challenges might the person face in exploring their newfound superpower?” Or, “How might the person’s relationships with others be affected by their superpower?”

Remember that ChatGPT’s answers are generated by artificial intelligence and may not always be perfect or exactly what you want.

However, they can still be a great source of inspiration and help you start writing.

Must-Have GPTs Assistant

I recommend installing the WebBrowser Assistant created by the OpenAI Team. This tool allows you to add relevant Bing results to your ChatGPT prompts.

This assistant adds the first web results to your ChatGPT prompts for more accurate and up-to-date conversations.

Advertisement

It is very easy to install in only two clicks. (Click on Start Chat.)

Screenshot from ChatGPT, April 2024

For example, if I ask, “Who is Vincent Terrasi?,” ChatGPT has no answer.

With WebBrower Assistant, the assistant creates a new prompt with the first Bing results, and now ChatGPT knows who Vincent Terrasi is.

Enabling reverse prompt engineeringScreenshot from ChatGPT, March 2023

You can test other GPT assistants available in the GPTs search engine if you want to use Google results.

Master Reverse Prompt Engineering

ChatGPT can be an excellent tool for reverse engineering prompts because it generates natural and engaging responses to any given input.

By analyzing the prompts generated by ChatGPT, it is possible to gain insight into the model’s underlying thought processes and decision-making strategies.

One key benefit of using ChatGPT to reverse engineer prompts is that the model is highly transparent in its decision-making.

Advertisement

This means that the reasoning and logic behind each response can be traced, making it easier to understand how the model arrives at its conclusions.

Once you’ve done this a few times for different types of content, you’ll gain insight into crafting more effective prompts.

Prepare Your ChatGPT For Generating Prompts

First, activate the reverse prompt engineering.

  • Type the following prompt: “Enable Reverse Prompt Engineering? By Reverse Prompt Engineering I mean creating a prompt from a given text.”
Enabling reverse prompt engineeringScreenshot from ChatGPT, March 2023

ChatGPT is now ready to generate your prompt. You can test the product description in a new chatbot session and evaluate the generated prompt.

  • Type: “Create a very technical reverse prompt engineering template for a product description about iPhone 11.”
Reverse Prompt engineering via WebChatGPTScreenshot from ChatGPT, March 2023

The result is amazing. You can test with a full text that you want to reproduce. Here is an example of a prompt for selling a Kindle on Amazon.

  • Type: “Reverse Prompt engineer the following {product), capture the writing style and the length of the text :
    product =”
Reverse prompt engineering: Amazon productScreenshot from ChatGPT, March 2023

I tested it on an SEJ blog post. Enjoy the analysis – it is excellent.

  • Type: “Reverse Prompt engineer the following {text}, capture the tone and writing style of the {text} to include in the prompt :
    text = all text coming from https://www.searchenginejournal.com/google-bard-training-data/478941/”
Reverse prompt engineering an SEJ blog postScreenshot from ChatGPT, March 2023

But be careful not to use ChatGPT to generate your texts. It is just a personal assistant.

Go Deeper

Prompts and examples for SEO:

  • Keyword research and content ideas prompt: “Provide a list of 20 long-tail keyword ideas related to ‘local SEO strategies’ along with brief content topic descriptions for each keyword.”
  • Optimizing content for featured snippets prompt: “Write a 40-50 word paragraph optimized for the query ‘what is the featured snippet in Google search’ that could potentially earn the featured snippet.”
  • Creating meta descriptions prompt: “Draft a compelling meta description for the following blog post title: ’10 Technical SEO Factors You Can’t Ignore in 2024′.”

Important Considerations:

  • Always Fact-Check: While ChatGPT can be a helpful tool, it’s crucial to remember that it may generate inaccurate or fabricated information. Always verify any facts, statistics, or quotes generated by ChatGPT before incorporating them into your content.
  • Maintain Control and Creativity: Use ChatGPT as a tool to assist your writing, not replace it. Don’t rely on it to do your thinking or create content from scratch. Your unique perspective and creativity are essential for producing high-quality, engaging content.
  • Iteration is Key: Refine and revise the outputs generated by ChatGPT to ensure they align with your voice, style, and intended message.

Additional Prompts for Rewording and SEO:
– Rewrite this sentence to be more concise and impactful.
– Suggest alternative phrasing for this section to improve clarity.
– Identify opportunities to incorporate relevant internal and external links.
– Analyze the keyword density and suggest improvements for better SEO.

Remember, while ChatGPT can be a valuable tool, it’s essential to use it responsibly and maintain control over your content creation process.

Experiment And Refine Your Prompting Techniques

Writing effective prompts for ChatGPT is an essential skill for any SEO professional who wants to harness the power of AI-generated content.

Advertisement

Hopefully, the insights and examples shared in this article can inspire you and help guide you to crafting stronger prompts that yield high-quality content.

Remember to experiment with layering prompts, iterating on the output, and continually refining your prompting techniques.

This will help you stay ahead of the curve in the ever-changing world of SEO.

More resources: 


Featured Image: Tapati Rinchumrus/Shutterstock

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS