Connect with us

SEO

Does Google Have A Problem With Big Robots.txt Files?

Published

on

Does Google Have A Problem With Big Robots.txt Files?

Google addresses the subject of robots.txt files and whether it’s a good SEO practice to keep them within a reasonable size.

This topic is discussed by Google’s Search Advocate John Mueller during the Google Search Central SEO office-hours hangout recorded on January 14.

David Zieger, an SEO manager for a large news publisher in Germany, joins the livestream with concerns about a “huge” and “complex” robots.txt file.

How huge are we talking here?

Zieger says there’s over 1,500 lines with a “multitude” of disallows that keeps growing over the years.

Advertisement

The disallows prevent Google from indexing HTML fragments and URLs where AJAX calls are used.

Zieger says it’s not possible to set a noindex, which is another way to keep the fragments and URLs out of Google’s index, so he’s resorted to filling the site’s robots.txt with disallows.

Are there any negative SEO effects that can result from a huge robots.txt file?

Here’s what Mueller says.

SEO Considerations For Large Robots.txt Files

A large robots.txt file will not directly cause any negative impact to a site’s SEO.

However, a large file is harder to maintain, which may lead to accidental issues down the road.

Advertisement

Mueller explains:

“No direct negative SEO issues with that, but it makes it a lot harder to maintain. And it makes it a lot easier to accidentally push something that does cause issues.

So just because it’s a large file doesn’t mean it’s a problem, but it makes it easier for you to create problems.”

Zieger follows up by asking if there are any issues with not including a sitemap in the robots.txt file.

Mueller says that’s not a problem:

“No. Those different ways of submitting a sitemap are all equivalent for us.”

Zieger then launches into a several more follow-up questions that we’ll take a look at in the next section.

Does Google Recognize HTML Fragments?

Zieger asks Mueller what would be the SEO impact of radically shortening the robots.txt file. Such as removing all the disallows, for example.

Advertisement

The following questions are asked:

  • Does Google recognize HTML fragments that aren’t relevant to site visitors?
  • Would HTML fragments end up in Google’s search index if they weren’t disallowed in robots.txt?
  • How does Google deal with pages where AJAX calls are used? (Such as a header or footer element)

He sums up his questions by stating most of what’s disallowed in his robots.txt file are header and footer elements that aren’t interesting for the user.

Mueller says it’s difficult to know exactly what would happen if those fragments were suddenly allowed to be indexed.

A trial and error approach might be the best way of figuring this out, Mueller explains:

“It’s hard to say what you mean with regards to those fragments

My thought there would be to try to figure out how those fragment URLs are used. And if you’re unsure, maybe take one of these fragment URLs and allow its crawling, and look at the content of that fragment URL, and then check to see what happens in search.

Does it affect anything with regards to the indexed content on your site?
Is some of that content findable within your site suddenly?
Is that a problem or not?

And try to work based on that, because it’s very easy to block things by robots.txt, which actually are not used for indexing, and then you spend a lot of time maintaining this big robots.txt file, but it actually doesn’t change that much for your website.”

Advertisement

Other Considerations For Building A Robots.txt File

Zieger has one last follow-up regarding robots.txt files, asking if there are any specific guidelines to follow when building one.

Mueller says there’s no specific format to follow:

“No, it’s essentially up to you. Like some sites have big files, some sites have small files, they should all just work.

We have an open source code of the robots.txt parser that we use. So what you can also do is get your developers to run that parser for you, or kind of set it up so that you can test it, and then check the URLs on your website with that parser to see which URLs would actually get blocked and what that would change. And that way you can test things before you make them live.”

The robots.txt parser Mueller refers to can be found on Github.

Hear the full discussion in the video below:

Advertisement

Featured Image: Screenshot from YouTube.com/GoogleSearchCentral, January 2022.




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

How To Write ChatGPT Prompts To Get The Best Results

Published

on

By

How To Write ChatGPT Prompts To Get The Best Results

ChatGPT is a game changer in the field of SEO. This powerful language model can generate human-like content, making it an invaluable tool for SEO professionals.

However, the prompts you provide largely determine the quality of the output.

To unlock the full potential of ChatGPT and create content that resonates with your audience and search engines, writing effective prompts is crucial.

In this comprehensive guide, we’ll explore the art of writing prompts for ChatGPT, covering everything from basic techniques to advanced strategies for layering prompts and generating high-quality, SEO-friendly content.

Writing Prompts For ChatGPT

What Is A ChatGPT Prompt?

A ChatGPT prompt is an instruction or discussion topic a user provides for the ChatGPT AI model to respond to.

Advertisement

The prompt can be a question, statement, or any other stimulus to spark creativity, reflection, or engagement.

Users can use the prompt to generate ideas, share their thoughts, or start a conversation.

ChatGPT prompts are designed to be open-ended and can be customized based on the user’s preferences and interests.

How To Write Prompts For ChatGPT

Start by giving ChatGPT a writing prompt, such as, “Write a short story about a person who discovers they have a superpower.”

ChatGPT will then generate a response based on your prompt. Depending on the prompt’s complexity and the level of detail you requested, the answer may be a few sentences or several paragraphs long.

Use the ChatGPT-generated response as a starting point for your writing. You can take the ideas and concepts presented in the answer and expand upon them, adding your own unique spin to the story.

Advertisement

If you want to generate additional ideas, try asking ChatGPT follow-up questions related to your original prompt.

For example, you could ask, “What challenges might the person face in exploring their newfound superpower?” Or, “How might the person’s relationships with others be affected by their superpower?”

Remember that ChatGPT’s answers are generated by artificial intelligence and may not always be perfect or exactly what you want.

However, they can still be a great source of inspiration and help you start writing.

Must-Have GPTs Assistant

I recommend installing the WebBrowser Assistant created by the OpenAI Team. This tool allows you to add relevant Bing results to your ChatGPT prompts.

This assistant adds the first web results to your ChatGPT prompts for more accurate and up-to-date conversations.

Advertisement

It is very easy to install in only two clicks. (Click on Start Chat.)

Screenshot from ChatGPT, April 2024

For example, if I ask, “Who is Vincent Terrasi?,” ChatGPT has no answer.

With WebBrower Assistant, the assistant creates a new prompt with the first Bing results, and now ChatGPT knows who Vincent Terrasi is.

Enabling reverse prompt engineeringScreenshot from ChatGPT, March 2023

You can test other GPT assistants available in the GPTs search engine if you want to use Google results.

Master Reverse Prompt Engineering

ChatGPT can be an excellent tool for reverse engineering prompts because it generates natural and engaging responses to any given input.

By analyzing the prompts generated by ChatGPT, it is possible to gain insight into the model’s underlying thought processes and decision-making strategies.

One key benefit of using ChatGPT to reverse engineer prompts is that the model is highly transparent in its decision-making.

Advertisement

This means that the reasoning and logic behind each response can be traced, making it easier to understand how the model arrives at its conclusions.

Once you’ve done this a few times for different types of content, you’ll gain insight into crafting more effective prompts.

Prepare Your ChatGPT For Generating Prompts

First, activate the reverse prompt engineering.

  • Type the following prompt: “Enable Reverse Prompt Engineering? By Reverse Prompt Engineering I mean creating a prompt from a given text.”
Enabling reverse prompt engineeringScreenshot from ChatGPT, March 2023

ChatGPT is now ready to generate your prompt. You can test the product description in a new chatbot session and evaluate the generated prompt.

  • Type: “Create a very technical reverse prompt engineering template for a product description about iPhone 11.”
Reverse Prompt engineering via WebChatGPTScreenshot from ChatGPT, March 2023

The result is amazing. You can test with a full text that you want to reproduce. Here is an example of a prompt for selling a Kindle on Amazon.

  • Type: “Reverse Prompt engineer the following {product), capture the writing style and the length of the text :
    product =”
Reverse prompt engineering: Amazon productScreenshot from ChatGPT, March 2023

I tested it on an SEJ blog post. Enjoy the analysis – it is excellent.

  • Type: “Reverse Prompt engineer the following {text}, capture the tone and writing style of the {text} to include in the prompt :
    text = all text coming from https://www.searchenginejournal.com/google-bard-training-data/478941/”
Reverse prompt engineering an SEJ blog postScreenshot from ChatGPT, March 2023

But be careful not to use ChatGPT to generate your texts. It is just a personal assistant.

Go Deeper

Prompts and examples for SEO:

  • Keyword research and content ideas prompt: “Provide a list of 20 long-tail keyword ideas related to ‘local SEO strategies’ along with brief content topic descriptions for each keyword.”
  • Optimizing content for featured snippets prompt: “Write a 40-50 word paragraph optimized for the query ‘what is the featured snippet in Google search’ that could potentially earn the featured snippet.”
  • Creating meta descriptions prompt: “Draft a compelling meta description for the following blog post title: ’10 Technical SEO Factors You Can’t Ignore in 2024′.”

Important Considerations:

  • Always Fact-Check: While ChatGPT can be a helpful tool, it’s crucial to remember that it may generate inaccurate or fabricated information. Always verify any facts, statistics, or quotes generated by ChatGPT before incorporating them into your content.
  • Maintain Control and Creativity: Use ChatGPT as a tool to assist your writing, not replace it. Don’t rely on it to do your thinking or create content from scratch. Your unique perspective and creativity are essential for producing high-quality, engaging content.
  • Iteration is Key: Refine and revise the outputs generated by ChatGPT to ensure they align with your voice, style, and intended message.

Additional Prompts for Rewording and SEO:
– Rewrite this sentence to be more concise and impactful.
– Suggest alternative phrasing for this section to improve clarity.
– Identify opportunities to incorporate relevant internal and external links.
– Analyze the keyword density and suggest improvements for better SEO.

Remember, while ChatGPT can be a valuable tool, it’s essential to use it responsibly and maintain control over your content creation process.

Experiment And Refine Your Prompting Techniques

Writing effective prompts for ChatGPT is an essential skill for any SEO professional who wants to harness the power of AI-generated content.

Advertisement

Hopefully, the insights and examples shared in this article can inspire you and help guide you to crafting stronger prompts that yield high-quality content.

Remember to experiment with layering prompts, iterating on the output, and continually refining your prompting techniques.

This will help you stay ahead of the curve in the ever-changing world of SEO.

More resources: 


Featured Image: Tapati Rinchumrus/Shutterstock

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Measuring Content Impact Across The Customer Journey

Published

on

By

Measuring Content Impact Across The Customer Journey

Understanding the impact of your content at every touchpoint of the customer journey is essential – but that’s easier said than done. From attracting potential leads to nurturing them into loyal customers, there are many touchpoints to look into.

So how do you identify and take advantage of these opportunities for growth?

Watch this on-demand webinar and learn a comprehensive approach for measuring the value of your content initiatives, so you can optimize resource allocation for maximum impact.

You’ll learn:

  • Fresh methods for measuring your content’s impact.
  • Fascinating insights using first-touch attribution, and how it differs from the usual last-touch perspective.
  • Ways to persuade decision-makers to invest in more content by showcasing its value convincingly.

With Bill Franklin and Oliver Tani of DAC Group, we unravel the nuances of attribution modeling, emphasizing the significance of layering first-touch and last-touch attribution within your measurement strategy. 

Check out these insights to help you craft compelling content tailored to each stage, using an approach rooted in first-hand experience to ensure your content resonates.

Advertisement

Whether you’re a seasoned marketer or new to content measurement, this webinar promises valuable insights and actionable tactics to elevate your SEO game and optimize your content initiatives for success. 

View the slides below or check out the full webinar for all the details.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

How to Find and Use Competitor Keywords

Published

on

How to Find and Use Competitor Keywords

Competitor keywords are the keywords your rivals rank for in Google’s search results. They may rank organically or pay for Google Ads to rank in the paid results.

Knowing your competitors’ keywords is the easiest form of keyword research. If your competitors rank for or target particular keywords, it might be worth it for you to target them, too.

There is no way to see your competitors’ keywords without a tool like Ahrefs, which has a database of keywords and the sites that rank for them. As far as we know, Ahrefs has the biggest database of these keywords.

How to find all the keywords your competitor ranks for

  1. Go to Ahrefs’ Site Explorer
  2. Enter your competitor’s domain
  3. Go to the Organic keywords report

The report is sorted by traffic to show you the keywords sending your competitor the most visits. For example, Mailchimp gets most of its organic traffic from the keyword “mailchimp.”

Mailchimp gets most of its organic traffic from the keyword, “mailchimp”.Mailchimp gets most of its organic traffic from the keyword, “mailchimp”.

Since you’re unlikely to rank for your competitor’s brand, you might want to exclude branded keywords from the report. You can do this by adding a Keyword > Doesn’t contain filter. In this example, we’ll filter out keywords containing “mailchimp” or any potential misspellings:

Filtering out branded keywords in Organic keywords reportFiltering out branded keywords in Organic keywords report

If you’re a new brand competing with one that’s established, you might also want to look for popular low-difficulty keywords. You can do this by setting the Volume filter to a minimum of 500 and the KD filter to a maximum of 10.

Finding popular, low-difficulty keywords in Organic keywordsFinding popular, low-difficulty keywords in Organic keywords

How to find keywords your competitor ranks for, but you don’t

  1. Go to Competitive Analysis
  2. Enter your domain in the This target doesn’t rank for section
  3. Enter your competitor’s domain in the But these competitors do section
Competitive analysis reportCompetitive analysis report

Hit “Show keyword opportunities,” and you’ll see all the keywords your competitor ranks for, but you don’t.

Content gap reportContent gap report

You can also add a Volume and KD filter to find popular, low-difficulty keywords in this report.

Volume and KD filter in Content gapVolume and KD filter in Content gap

How to find keywords multiple competitors rank for, but you don’t

  1. Go to Competitive Analysis
  2. Enter your domain in the This target doesn’t rank for section
  3. Enter the domains of multiple competitors in the But these competitors do section
Competitive analysis report with multiple competitorsCompetitive analysis report with multiple competitors

You’ll see all the keywords that at least one of these competitors ranks for, but you don’t.

Content gap report with multiple competitorsContent gap report with multiple competitors

You can also narrow the list down to keywords that all competitors rank for. Click on the Competitors’ positions filter and choose All 3 competitors:

Selecting all 3 competitors to see keywords all 3 competitors rank forSelecting all 3 competitors to see keywords all 3 competitors rank for
  1. Go to Ahrefs’ Site Explorer
  2. Enter your competitor’s domain
  3. Go to the Paid keywords report
Paid keywords reportPaid keywords report

This report shows you the keywords your competitors are targeting via Google Ads.

Since your competitor is paying for traffic from these keywords, it may indicate that they’re profitable for them—and could be for you, too.

Advertisement

You know what keywords your competitors are ranking for or bidding on. But what do you do with them? There are basically three options.

1. Create pages to target these keywords

You can only rank for keywords if you have content about them. So, the most straightforward thing you can do for competitors’ keywords you want to rank for is to create pages to target them.

However, before you do this, it’s worth clustering your competitor’s keywords by Parent Topic. This will group keywords that mean the same or similar things so you can target them all with one page.

Here’s how to do that:

  1. Export your competitor’s keywords, either from the Organic Keywords or Content Gap report
  2. Paste them into Keywords Explorer
  3. Click the “Clusters by Parent Topic” tab
Clustering keywords by Parent TopicClustering keywords by Parent Topic

For example, MailChimp ranks for keywords like “what is digital marketing” and “digital marketing definition.” These and many others get clustered under the Parent Topic of “digital marketing” because people searching for them are all looking for the same thing: a definition of digital marketing. You only need to create one page to potentially rank for all these keywords.

Keywords under the cluster of "digital marketing"Keywords under the cluster of "digital marketing"

2. Optimize existing content by filling subtopics

You don’t always need to create new content to rank for competitors’ keywords. Sometimes, you can optimize the content you already have to rank for them.

How do you know which keywords you can do this for? Try this:

Advertisement
  1. Export your competitor’s keywords
  2. Paste them into Keywords Explorer
  3. Click the “Clusters by Parent Topic” tab
  4. Look for Parent Topics you already have content about

For example, if we analyze our competitor, we can see that seven keywords they rank for fall under the Parent Topic of “press release template.”

Our competitor ranks for seven keywords that fall under the "press release template" clusterOur competitor ranks for seven keywords that fall under the "press release template" cluster

If we search our site, we see that we already have a page about this topic.

Site search finds that we already have a blog post on press release templatesSite search finds that we already have a blog post on press release templates

If we click the caret and check the keywords in the cluster, we see keywords like “press release example” and “press release format.”

Keywords under the cluster of "press release template"Keywords under the cluster of "press release template"

To rank for the keywords in the cluster, we can probably optimize the page we already have by adding sections about the subtopics of “press release examples” and “press release format.”

3. Target these keywords with Google Ads

Paid keywords are the simplest—look through the report and see if there are any relevant keywords you might want to target, too.

For example, Mailchimp is bidding for the keyword “how to create a newsletter.”

Mailchimp is bidding for the keyword “how to create a newsletter”Mailchimp is bidding for the keyword “how to create a newsletter”

If you’re ConvertKit, you may also want to target this keyword since it’s relevant.

If you decide to target the same keyword via Google Ads, you can hover over the magnifying glass to see the ads your competitor is using.

Mailchimp's Google Ad for the keyword “how to create a newsletter”Mailchimp's Google Ad for the keyword “how to create a newsletter”

You can also see the landing page your competitor directs ad traffic to under the URL column.

The landing page Mailchimp is directing traffic to for “how to create a newsletter”The landing page Mailchimp is directing traffic to for “how to create a newsletter”

Learn more

Check out more tutorials on how to do competitor keyword analysis:

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS