Connect with us

SEO

SEO Community Support For Ukraine & How You Can Help

Published

on

SEO Community Support For Ukraine & How You Can Help

with Brock Cooper

Ukrainian streets once filled with the laughter of children and hurried walks of people heading to work are now abandoned.

Workers of all kinds have had to abandon their posts to flee their homes or worse – to take up arms and defend their families.

Russia’s invasion has ground life as they knew it to a halt for millions of Ukrainians.

Outpourings of support from the search engine optimization and digital marketing industry abound as many companies and employees hail from the besieged country.

Tech is a $6.8 billion business in Ukraine and thousands of its 4.4 million residents work in the sector.

This invasion hits very close to many of us, even here at Search Engine Journal where the family of Loren Baker, SEJ’s Founder, is currently fighting to defend the people and the land of Ukraine.

The digital marketing and SEO community is a family. And when members of our family are hurting, we provide support in the best ways we know how. We will be adding to this list with updates and additional ways to help.

Google Donating $15 Million & Fighting Disinformation

Google and its employees are donating a combined $15 million in money and in-kind support to aid relief.

The search giant will also donate $5 million in advertising credits to humanitarian and intergovernmental organizations it considers trustworthy.

“In this extraordinary crisis, we are taking extraordinary measures to stop the spread of misinformation and disrupt disinformation campaigns online,” Google President of Global Affairs Kent Walker wrote in a March 1 statement.

“We will continue to monitor the situation and take additional actions as needed — and we join the international community in expressing sincere hope for a return to a peaceful and sovereign Ukraine,” he added.

Google’s announcement also notes that YouTube channels from RT and Sputnik are now blocked across Europe.

Russian state-funded media have had their monetization options across all Google platforms frozen for the time being, and RT is banned from advertising entirely.

Walker also notes that products like Google Pay may become unavailable in some countries as Google strives to comply with government sanctions.

Canceled Conference Event Donates $35,000

Organizer Christoph Cemper added, “We had speakers from Ukraine. LRTcon was headlined by some amazing speakers from Ukraine. We had guests from Ukraine. LRTcon was visited by many competent and nice people from Ukraine… I am very proud about being able to help to so quick and so big. And with this, the story of LRTcon ends.”

Conductor To Pay Salaries Of Drafted Employees To Family

Seth Besmertnik of Conductor provided updates on LinkedIn this week on employee Oleksander Lytvyn, who was risking his life patrolling the streets while his family hunkers down in a parking garage.

“Our Ukraine team has over and over proved to be loyal, dedicated, proud, and open to always getting better.” Besmertnik wrote.

Conductor saw the writing on the wall more than a month ago as Russia provoked and threatened the country, Besmertnik wrote. They provided assistance and incentives for their Ukrainian team to escape.

Half of them chose to flee, but the other half stayed.

They message team members in Ukraine every 12 hours offering monetary and other assistance.

“For those who have stayed, we let them know what if they get drafted, we will provide income to their families as long as they are in service,” Besmertnik wrote.

SERPstats Sends Blog Editor To Volunteer Journalists Initiative

All-in-one SEO platform SERPstat, based in Ukraine, tweeted March 2 that Taras Prystatsky, its Blog Editor, had joined the Ukrainian Volunteer Journalists Initiative.

Today, the company tweeted:

How SEO Pros Can Help Our Colleagues In Ukraine

Hire a Ukrainian Remote Worker

Olesia Korobka has organized a makeshift database of Ukrainian SEO and digital marketing professionals who are currently looking for remote work.

SEO consultant Alexis Rylko has compiled a post of ‘Awesome SEO & Digital Marketing Tools Made In Ukraine‘ for those who might like to support the cause with their business.

Support Ukrainian Businesses

Julia Nesterets, CEO of crawler and log analyzer tool Jet Octopus, sent a tweet from her vehicle as she and her newborn baby delivered medical supplies, fundraised, and gave demos from the road as she works remote:

Ahrefs’ CEO Dmitry Gerasimenko is Ukrainian and offered to double subscriptions of anyone that donates to organizations that support the peace in Ukraine.

If a person donates the amount of a six-month plan, Ahrefs will support them with an entire year of service.

SE Ranking has a message on their website supporting Ukraine and provides links to how people can help.

One points to HelpUkraineWin.org, a resource created by Tumblr’s head of product, Bohdan Kit. There, he shares a form for one-time or monthly donations, a list of ways to donate or volunteer in other ways, and a feed from #StandWithUkraine on Twitter.

SERanking’s head of content Svetlana Shchehel, also an author at Search Engine Journal, recommends that people donate to Unicef to provide much-needed emergency relief for Ukrainian children.

Olga Andrienko, VP of brand marketing at Semrush, tweeted, “We strongly oppose any act of war, including the Russian invasion of Ukraine. Prowly, our Polish entity, is coordinating an effort to provide some direct relief to people in need within Ukraine via NGO support.”

Any non-governmental organizations wishing to support their efforts can contact them directly at [email protected].

Digital marketing agency Switch out of Malta has a page on their website dedicated to Ukraine support and every team member is donated one hour per day to humanitarian causes. There is also a contact form where people can request or offer services to help the people of Ukraine.

Fact-Check Everything

Information and misinformation about the Ukrainian conflict are everywhere, but a band of public relations specialists located in Kyiv wants to make sure the world hears only the truth.

More than 100 PR professionals created a Google form where media can ask for first-hand accounts from experts, volunteers, and eyewitnesses, according to a press release. They can also receive pictures and videos of the war.

Before making any donations or services to support Ukraine, make sure the charity is trustworthy.

Often people take advantage of strife and disaster to make money. There are others that have good intentions but support the cause illegally.

If you are looking for the most trusted place to donate, visit the Support Ukraine page from the Ministry of Foreign Affairs of Ukraine.

There are also requests circulating for people to participate in DDOS attacks to take down Russian sites.

It must be noted that not only is this illegal, but a well-intentioned person could easily end up participating in something else entirely.

Hope For Quick Resolution

The hearts and minds of the world are on the people of Ukraine as they continue to fight for their freedom against overwhelming odds.

Acrid smoke permeates from the husks of bombed-out buildings and gunfire rings out in the streets as Russian forces take on not only the Ukrainian military but citizens as well.

Search Engine Journal fully supports the people of Ukraine in their time of need and encourages everyone to do what they can to help the thousands of innocents caught in the middle.

With research and notes from Brock Cooper, News Writer.


Featured Image: DOERS/Shutterstock




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

How To Use ChatGPT For Keyword Research

Published

on

By

How To Use ChatGPT For Keyword Research

Anyone not using ChatGPT for keyword research is missing a trick.

You can save time and understand an entire topic in seconds instead of hours.

In this article, I outline my most effective ChatGPT prompts for keyword research and teach you how I put them together so that you, too, can take, edit, and enhance them even further.

But before we jump into the prompts, I want to emphasize that you shouldn’t replace keyword research tools or disregard traditional keyword research methods.

ChatGPT can make mistakes. It can even create new keywords if you give it the right prompt. For example, I asked it to provide me with a unique keyword for the topic “SEO” that had never been searched before.

Interstellar Internet SEO: Optimizing content for the theoretical concept of an interstellar internet, considering the challenges of space-time and interplanetary communication delays.”

Although I want to jump into my LinkedIn profile and update my title to “Interstellar Internet SEO Consultant,” unfortunately, no one has searched that (and they probably never will)!

You must not blindly rely on the data you get back from ChatGPT.

What you can rely on ChatGPT for is the topic ideation stage of keyword research and inspiration.

ChatGPT is a large language model trained with massive amounts of data to accurately predict what word will come next in a sentence. However, it does not know how to do keyword research yet.

Instead, think of ChatGPT as having an expert on any topic armed with the information if you ask it the right question.

In this guide, that is exactly what I aim to teach you how to do – the most essential prompts you need to know when performing topical keyword research.

Best ChatGPT Keyword Research Prompts

The following ChatGPT keyword research prompts can be used on any niche, even a topic to which you are brand new.

For this demonstration, let’s use the topic of “SEO” to demonstrate these prompts.

Generating Keyword Ideas Based On A Topic

What Are The {X} Most Popular Sub-topics Related To {Topic}?

Screenshot from ChatGPT 4, April 2024

The first prompt is to give you an idea of the niche.

As shown above, ChatGPT did a great job understanding and breaking down SEO into three pillars: on-page, off-page & technical.

The key to the following prompt is to take one of the topics ChatGPT has given and query the sub-topics.

What Are The {X} Most Popular Sub-topics Related To {Sub-topic}?

For this example, let’s query, “What are the most popular sub-topics related to keyword research?”

Having done keyword research for over 10 years, I would expect it to output information related to keyword research metrics, the types of keywords, and intent.

Let’s see.

ChatGPT keyword prompt subtopicScreenshot from ChatGPT 4, April 2024

Again, right on the money.

To get the keywords you want without having ChatGPT describe each answer, use the prompt “list without description.”

Here is an example of that.

List Without Description The Top {X} Most Popular Keywords For The Topic Of {X}chatgpt keyword research prompt for most popular keywords

You can even branch these keywords out further into their long-tail.

Example prompt:

List Without Description The Top {X} Most Popular Long-tail Keywords For The Topic “{X}”

chatgpt keyword research prompt longtail keywordsScreenshot ChatGPT 4,April 2024

List Without Description The Top Semantically Related Keywords And Entities For The Topic {X}

You can even ask ChatGPT what any topic’s semantically related keywords and entities are!

chatgpt keyword research semantic intentScreenshot ChatGPT 4, April 2024

Tip: The Onion Method Of Prompting ChatGPT

When you are happy with a series of prompts, add them all to one prompt. For example, so far in this article, we have asked ChatGPT the following:

  • What are the four most popular sub-topics related to SEO?
  • What are the four most popular sub-topics related to keyword research
  • List without description the top five most popular keywords for “keyword intent”?
  • List without description the top five most popular long-tail keywords for the topic “keyword intent types”?
  • List without description the top semantically related keywords and entities for the topic “types of keyword intent in SEO.”

Combine all five into one prompt by telling ChatGPT to perform a series of steps. Example:

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, Step 4, and Step 5”

Example:

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, Step 4 and Step 5. Step 1 – Generate an answer for the 3 most popular sub-topics related to {Topic}?. Step 2 – Generate 3 of the most popular sub-topics related to each answer. Step 3 – Take those answers and list without description their top 3 most popular keywords. Step 4 – For the answers given of their most popular keywords, provide 3 long-tail keywords. Step 5 – for each long-tail keyword offered in the response, a list without descriptions 3 of their top semantically related keywords and entities.”

Generating Keyword Ideas Based On A Question

Taking the steps approach from above, we can get ChatGPT to help streamline getting keyword ideas based on a question. For example, let’s ask, “What is SEO?

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, and Step 4. Step 1 Generate 10 questions about “{Question}”?. Step 2 – Generate 5 more questions about “{Question}” that do not repeat the above. Step 3 – Generate 5 more questions about “{Question}” that do not repeat the above. Step 4 – Based on the above Steps 1,2,3 suggest a final list of questions avoiding duplicates or semantically similar questions.”

chatgpt for question keyword researchScreenshot ChatGPT 4, April 2024

Generating Keyword Ideas Using ChatGPT Based On The Alphabet Soup Method

One of my favorite methods, manually, without even using a keyword research tool, is to generate keyword research ideas from Google autocomplete, going from A to Z.

Generating Keyword Ideas using ChatGPT Based on the Alphabet Soup MethodScreenshot from Google autocomplete, April 2024

You can also do this using ChatGPT.

Example prompt:

“give me popular keywords that includes the keyword “SEO”, and the next letter of the word starts with a”

ChatGPT Alphabet keyword research methodScreenshot from ChatGPT 4, April 2024

Tip: Using the onion prompting method above, we can combine all this in one prompt.

“Give me five popular keywords that include “SEO” in the word, and the following letter starts with a. Once the answer has been done, move on to giving five more popular keywords that include “SEO” for each letter of the alphabet b to z.”

Generating Keyword Ideas Based On User Personas

When it comes to keyword research, understanding user personas is essential for understanding your target audience and keeping your keyword research focused and targeted. ChatGPT may help you get an initial understanding of customer personas.

Example prompt:

“For the topic of “{Topic}” list 10 keywords each for the different types of user personas”

ChatGPT and user personasScreenshot from ChatGPT 4, April 2024

You could even go a step further and ask for questions based on those topics that those specific user personas may be searching for:

ChatGPT and keyword research based on personaScreenshot ChatGPT 4, April 2024

As well as get the keywords to target based on those questions:

“For each question listed above for each persona, list the keywords, as well as the long-tail keywords to target, and put them in a table”

question and longtail and user persona using a table for ChatGPT keyword researchScreenshot from ChatGPT 4, April 2024

Generating Keyword Ideas Using ChatGPT Based On Searcher Intent And User Personas

Understanding the keywords your target persona may be searching is the first step to effective keyword research. The next step is to understand the search intent behind those keywords and which content format may work best.

For example, a business owner who is new to SEO or has just heard about it may be searching for “what is SEO.”

However, if they are further down the funnel and in the navigational stage, they may search for “top SEO firms.”

You can query ChatGPT to inspire you here based on any topic and your target user persona.

SEO Example:

“For the topic of “{Topic}” list 10 keywords each for the different types of searcher intent that a {Target Persona} would be searching for”

ChatGPT For Keyword Research Admin

Here is how you can best use ChatGPT for keyword research admin tasks.

Using ChatGPT As A Keyword Categorization Tool

One of the use cases for using ChatGPT is for keyword categorization.

In the past, I would have had to devise spreadsheet formulas to categorize keywords or even spend hours filtering and manually categorizing keywords.

ChatGPT can be a great companion for running a short version of this for you.

Let’s say you have done keyword research in a keyword research tool, have a list of keywords, and want to categorize them.

You could use the following prompt:

“Filter the below list of keywords into categories, target persona, searcher intent, search volume and add information to a six-column table: List of keywords – [LIST OF KEYWORDS], Keyword Search Volume [SEARCH VOLUMES] and Keyword Difficulties [KEYWORD DIFFICUTIES].”

Using Chat GPT as a Keyword Categorization ToolScreenshot from ChatGPT, April 2024

Tip: Add keyword metrics from the keyword research tools, as using the search volumes that a ChatGPT prompt may give you will be wildly inaccurate at best.

Using ChatGPT For Keyword Clustering

Another of ChatGPT’s use cases for keyword research is to help you cluster. Many keywords have the same intent, and by grouping related keywords, you may find that one piece of content can often target multiple keywords at once.

However, be careful not to rely only on LLM data for clustering. What ChatGPT may cluster as a similar keyword, the SERP or the user may not agree with. But it is a good starting point.

The big downside of using ChatGPT for keyword clustering is actually the amount of keyword data you can cluster based on the memory limits.

So, you may find a keyword clustering tool or script that is better for large keyword clustering tasks. But for small amounts of keywords, ChatGPT is actually quite good.

A great use small keyword clustering use case using ChatGPT is for grouping People Also Ask (PAA) questions.

Use the following prompt to group keywords based on their semantic relationships. For example:

“Organize the following keywords into groups based on their semantic relationships, and give a short name to each group: [LIST OF PAA], create a two-column table where each keyword sits on its own row.

Using Chat GPT For Keyword ClusteringScreenshot from ChatGPT, April 2024

Using Chat GPT For Keyword Expansion By Patterns

One of my favorite methods of doing keyword research is pattern spotting.

Most seed keywords have a variable that can expand your target keywords.

Here are a few examples of patterns:

1. Question Patterns

(who, what, where, why, how, are, can, do, does, will)

“Generate [X] keywords for the topic “[Topic]” that contain any or all of the following “who, what, where, why, how, are, can, do, does, will”

question based keywords keyword research ChatGPTScreenshot ChatGPT 4, April 2024

2. Comparison Patterns

Example:

“Generate 50 keywords for the topic “{Topic}” that contain any or all of the following “for, vs, alternative, best, top, review”

chatgpt comparison patterns for keyword researchScreenshot ChatGPT 4, April 2024

3. Brand Patterns

Another one of my favorite modifiers is a keyword by brand.

We are probably all familiar with the most popular SEO brands; however, if you aren’t, you could ask your AI friend to do the heavy lifting.

Example prompt:

“For the top {Topic} brands what are the top “vs” keywords”

ChatGPT brand patterns promptScreenshot ChatGPT 4, April 2024

4. Search Intent Patterns

One of the most common search intent patterns is “best.”

When someone is searching for a “best {topic}” keyword, they are generally searching for a comprehensive list or guide that highlights the top options, products, or services within that specific topic, along with their features, benefits, and potential drawbacks, to make an informed decision.

Example:

“For the topic of “[Topic]” what are the 20 top keywords that include “best”

ChatGPT best based keyword researchScreenshot ChatGPT 4, April 2024

Again, this guide to keyword research using ChatGPT has emphasized the ease of generating keyword research ideas by utilizing ChatGPT throughout the process.

Keyword Research Using ChatGPT Vs. Keyword Research Tools

Free Vs. Paid Keyword Research Tools

Like keyword research tools, ChatGPT has free and paid options.

However, one of the most significant drawbacks of using ChatGPT for keyword research alone is the absence of SEO metrics to help you make smarter decisions.

To improve accuracy, you could take the results it gives you and verify them with your classic keyword research tool – or vice versa, as shown above, uploading accurate data into the tool and then prompting.

However, you must consider how long it takes to type and fine-tune your prompt to get your desired data versus using the filters within popular keyword research tools.

For example, if we use a popular keyword research tool using filters, you could have all of the “best” queries with all of their SEO metrics:

ahrefs screenshot for best seoScreenshot from Ahrefs Keyword Explorer, March 2024

And unlike ChatGPT, generally, there is no token limit; you can extract several hundred, if not thousands, of keywords at a time.

As I have mentioned multiple times throughout this piece, you cannot blindly trust the data or SEO metrics it may attempt to provide you with.

The key is to validate the keyword research with a keyword research tool.

ChatGPT For International SEO Keyword Research

ChatGPT can be a terrific multilingual keyword research assistant.

For example, if you wanted to research keywords in a foreign language such as French. You could ask ChatGPT to translate your English keywords;

translating keywords with ChatGPTScreenshot ChatGPT 4, Apil 2024
The key is to take the data above and paste it into a popular keyword research tool to verify.
As you can see below, many of the keyword translations for the English keywords do not have any search volume for direct translations in French.
verifying the data with ahrefsScreenshot from Ahrefs Keyword Explorer, April 2024

But don’t worry, there is a workaround: If you have access to a competitor keyword research tool, you can see what webpage is ranking for that query – and then identify the top keyword for that page based on the ChatGPT translated keywords that do have search volume.

top keyword from ahrefs keyword explorerScreenshot from Ahrefs Keyword Explorer, April 2024

Or, if you don’t have access to a paid keyword research tool, you could always take the top-performing result, extract the page copy, and then ask ChatGPT what the primary keyword for the page is.

Key Takeaway

ChatGPT can be an expert on any topic and an invaluable keyword research tool. However, it is another tool to add to your toolbox when doing keyword research; it does not replace traditional keyword research tools.

As shown throughout this tutorial, from making up keywords at the beginning to inaccuracies around data and translations, ChatGPT can make mistakes when used for keyword research.

You cannot blindly trust the data you get back from ChatGPT.

However, it can offer a shortcut to understanding any topic for which you need to do keyword research and, as a result, save you countless hours.

But the key is how you prompt.

The prompts I shared with you above will help you understand a topic in minutes instead of hours and allow you to better seed keywords using keyword research tools.

It can even replace mundane keyword clustering tasks that you used to do with formulas in spreadsheets or generate ideas based on keywords you give it.

Paired with traditional keyword research tools, ChatGPT for keyword research can be a powerful tool in your arsenal.

More resources:


Featured Image: Tatiana Shepeleva/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Hints At Improving Site Rankings In Next Update

Published

on

By

Google Hints At Improving Site Rankings In Next Update

Google’s John Mueller says the Search team is “explicitly evaluating” how to reward sites that produce helpful, high-quality content when the next core update rolls out.

The comments came in response to a discussion on X about the impact of March’s core update and September’s helpful content update.

In a series of tweets, Mueller acknowledged the concerns, stating:

“I imagine for most sites strongly affected, the effects will be site-wide for the time being, and it will take until the next update to see similar strong effects (assuming the new state of the site is significantly better than before).”

He added:

“I can’t make any promises, but the team working on this is explicitly evaluating how sites can / will improve in Search for the next update. It would be great to show more users the content that folks have worked hard on, and where sites have taken helpfulness to heart.”

What Does This Mean For SEO Professionals & Site Owners?

Mueller’s comments confirm Google is aware of critiques about the March core update and is refining its ability to identify high-quality sites and reward them appropriately in the next core update.

For websites, clearly demonstrating an authentic commitment to producing helpful and high-quality content remains the best strategy for improving search performance under Google’s evolving systems.

The Aftermath Of Google’s Core Updates

Google’s algorithm updates, including the September “Helpful Content Update” and the March 2024 update, have far-reaching impacts on rankings across industries.

While some sites experienced surges in traffic, others faced substantial declines, with some reporting visibility losses of up to 90%.

As website owners implement changes to align with Google’s guidelines, many question whether their efforts will be rewarded.

There’s genuine concern about the potential for long-term or permanent demotions for affected sites.

Recovery Pathway Outlined, But Challenges Remain

In a previous statement, Mueller acknowledged the complexity of the recovery process, stating that:

“some things take much longer to be reassessed (sometimes months, at the moment), and some bigger effects require another update cycle.”

Mueller clarified that not all changes would require a new update cycle but cautioned that “stronger effects will require another update.”

While affirming that permanent changes are “not very useful in a dynamic world,” Mueller adds that “recovery” implies a return to previous levels, which may be unrealistic given evolving user expectations.

“It’s never ‘just-as-before’,” Mueller stated.

Improved Rankings On The Horizon?

Despite the challenges, Mueller has offered glimmers of hope for impacted sites, stating:

“Yes, sites can grow again after being affected by the ‘HCU’ (well, core update now). This isn’t permanent. It can take a lot of work, time, and perhaps update cycles, and/but a different – updated – site will be different in search too.”

He says the process may require “deep analysis to understand how to make a website relevant in a modern world, and significant work to implement those changes — assuming that it’s something that aligns with what the website even wants.”

Looking Ahead

Google’s search team is actively working on improving site rankings and addressing concerns with the next core update.

However, recovery requires patience, thorough analysis, and persistent effort.

The best way to spend your time until the next update is to remain consistent and produce the most exceptional content in your niche.


FAQ

How long does it generally take for a website to recover from the impact of a core update?

Recovery timelines can vary and depend on the extent and type of updates made to align with Google’s guidelines.

Google’s John Mueller noted that some changes might be reassessed quickly, while more substantial effects could take months and require additional update cycles.

Google acknowledges the complexity of the recovery process, indicating that significant improvements aligned with Google’s quality signals might be necessary for a more pronounced recovery.

What impact did the March and September updates have on websites, and what steps should site owners take?

The March and September updates had widespread effects on website rankings, with some sites experiencing traffic surges while others faced up to 90% visibility losses.

Publishing genuinely useful, high-quality content is key for website owners who want to bounce back from a ranking drop or maintain strong rankings. Stick to Google’s recommendations and adapt as they keep updating their systems.

To minimize future disruptions from algorithm changes, it’s a good idea to review your whole site thoroughly and build a content plan centered on what your users want and need.

Is it possible for sites affected by core updates to regain their previous ranking positions?

Sites can recover from the impact of core updates, but it requires significant effort and time.

Mueller suggested that recovery might happen over multiple update cycles and involves a deep analysis to align the site with current user expectations and modern search criteria.

While a return to previous levels isn’t guaranteed, sites can improve and grow by continually enhancing the quality and relevance of their content.


Featured Image: eamesBot/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Reveals Two New Web Crawlers

Published

on

By

Google Reveals Two New Web Crawlers

Google revealed details of two new crawlers that are optimized for scraping image and video content for “research and development” purposes. Although the documentation doesn’t explicitly say so, it’s presumed that there is no impact in ranking should publishers decide to block the new crawlers.

It should be noted that the data scraped by these crawlers are not explicitly for AI training data, that’s what the Google-Extended crawler is for.

GoogleOther Crawlers

The two new crawlers are versions of Google’s GoogleOther crawler that was launched in April 2023. The original GoogleOther crawler was also designated for use by Google product teams for research and development in what is described as one-off crawls, the description of which offers clues about what the new GoogleOther variants will be used for.

The purpose of the original GoogleOther crawler is officially described as:

“GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.”

Two GoogleOther Variants

There are two new GoogleOther crawlers:

  • GoogleOther-Image
  • GoogleOther-Video

The new variants are for crawling binary data, which is data that’s not text. HTML data is generally referred to as text files, ASCII or Unicode files. If it can be viewed in a text file then it’s a text file/ASCII/Unicode file. Binary files are files that can’t be open in a text viewer app, files like image, audio, and video.

The new GoogleOther variants are for image and video content. Google lists user agent tokens for both of the new crawlers which can be used in a robots.txt for blocking the new crawlers.

1. GoogleOther-Image

User agent tokens:

  • GoogleOther-Image
  • GoogleOther

Full user agent string:

GoogleOther-Image/1.0

2. GoogleOther-Video

User agent tokens:

  • GoogleOther-Video
  • GoogleOther

Full user agent string:

GoogleOther-Video/1.0

Newly Updated GoogleOther User Agent Strings

Google also updated the GoogleOther user agent strings for the regular GoogleOther crawler. For blocking purposes you can continue using the same user agent token as before (GoogleOther). The new Users Agent Strings are just the data sent to servers to identify the full description of the crawlers, in particular the technology used. In this case the technology used is Chrome, with the model number periodically updated to reflect which version is used (W.X.Y.Z is a Chrome version number placeholder in the example listed below)

The full list of GoogleOther user agent strings:

  • Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; GoogleOther)
  • Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36

GoogleOther Family Of Bots

These new bots may from time to time show up in your server logs and this information will help in identifying them as genuine Google crawlers and will help publishers who may want to opt out of having their images and videos scraped for research and development purposes.

Read the updated Google crawler documentation

GoogleOther-Image

GoogleOther-Video

Featured Image by Shutterstock/ColorMaker

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending