SEO tracking involves regularly checking a set of metrics to evaluate a website’s performance in search engine results. Some of the most widely adopted metrics include keyword rankings, organic traffic, conversions, and referring domain growth.

Tracking the right metrics is crucial for SEO (search engine optimization) success. You need them to analyze your SEO performance, report to stakeholders, and take the right kind of action to improve your site’s visibility (such as improving content or building more backlinks).

Besides keeping an eye on your own website’s key metrics, it’s also smart to check out how your competitors are doing on the same metrics as you. If you notice they’re getting good results, you can figure out what tactics they’re using and consider using them too.

You can track SEO for your site to a fair degree using free tools like Google Search Console or Ahrefs Webmaster Tools. If you want deeper insights, better data, and the ability to analyze your competitors’ websites, you’ll need a tool like Ahrefs.

This guide is aimed at getting you started with tracking your SEO progress the right way. We’ll cover:

• What metrics are worth tracking in SEO.
• How to set up the tools to get the data you need.
• How to track your competitors.
• How to go a step further and build an SEO report.

While there are numerous metrics and KPIs you could track, it’s not necessary to monitor all of them continuously. You really just need these seven key metrics to effectively gauge whether your SEO efforts are working.

1. Keyword rankings

Keyword ranking refers to where your page shows up on the search engine results page (SERP) for a specific keyword. It’s like a spot on a list, and you want your page to be as high up on that list as possible — the higher the spot, the more visitors you can attract.

It’s important to keep an eye on where your keywords are ranking because if they drop lower on the list, your website might get fewer visitors. But you don’t have to watch the rankings for every single keyword, just the main ones that matter most for your key pages.

Also, if you notice your rankings are climbing higher, that’s a good sign. It means that your SEO efforts are paying off.

How to track keyword rankings

To track your keyword rankings, it’s best to use a rank tracker tool like Ahrefs’ Rank Tracker; a tool that allows you to create a list of keywords and automatically monitor their positions in the SERPs for different locations, both for mobile and desktop.

Rank Tracker will suggest keywords for tracking when you set up a new project. Just make sure you’re tracking them in the locations you want to rank (that is, countries where you can serve clients and languages in which you create content).

No need to add each and every keyword from that list. Just add the ones that are important to you and you’ll likely want to track and improve. Typically, you’ll want to track target keywords — the main topic of the page and the main keyword you optimize for.

Once added, you can see your keywords in Rank Tracker’s Overview report.

Another way to start tracking keywords is to hit Add keywords in the top right corner — best for adding single keywords or importing a list from a document.

And once data starts rolling in, you will be able to see your ranking progress in time. In the screenshot below, the Ranking history report with a quick insight into recent ranking history and a full ranking history graph.

2. Share of voice

Share of voice (SOV) is a measure of how many clicks your website gets from search engines compared to the total number of clicks available for the keywords you’re tracking.

The higher your rankings, the higher your Share of Voice, and the larger your slice of the market pie.

SOV is a one-of-a-kind metric because of two things:

• It considers your performance in context to your competitors, giving you a more accurate picture of where you stand in your industry.
• It doesn’t take into account the search volume of keywords with all of their fluctuations. If you see that your traffic has gone down but your Share of Voice (SOV) remains high, it suggests that the lower traffic is because the keywords you’re targeting have become less popular overall, rather than a decrease in the effectiveness of your SEO strategies.

How to track share of voice

The share of voice metric is another reason to get a rank tracking tool. If the feature is supported, these kinds of tools calculate the metric automatically, so there’s no need to keep a spreadsheet with manually tracked numbers.

In Ahrefs’s Rank Tracker, you’ll find SOV under the Competitors tab.

SOV is calculated by taking all of the tracked keywords into account, yet some of your keywords might be more important than others. If that’s the case, you can track SOV only for a certain topic, SEO campaigns, specific authors, etc. Just select a set of keywords and assign a tag for them.

Then, simply select that tag in the Competitors report.

3. Organic traffic

Organic traffic is basically the number of clicks that come to your website from people finding it through Google. If your website shows up higher on the SERPs, usually more people will click on it and visit your site.

Keeping track of how many visitors come to your site from search engines helps you understand if what you’re doing with SEO is actually working. If you see more visitors over time, your SEO efforts are paying off.

Organic traffic is the pinnacle of SEO, but it’s also important to understand which keywords drive that traffic. So although it’s arguably the most important metric, it’s never a good idea to track this metric alone.

How to track organic traffic

In terms of raw organic traffic from Google Search, the most accurate data will likely come from their Search Console (for Bing, that would be Webmaster Tools). You can view this data right inside the tool or integrate it with analytics tools like Google Analytics, Hubspot, and Ahrefs for more convenience.

Raw traffic data is useful for getting a quick snapshot of your current performance, tracking growth trends, and calculating traffic growth for your reports.

But to dive a bit deeper into your organic traffic data, you might want to use a tool like Ahrefs’ Site Explorer because it makes it easier to analyze performance. Here are a few ways you can use the Overview and Top pages report in that tool.

Overlay competitor data on top for a quick performance analysis.

Overlay organic pages to see how adding new content correlates with traffic.

See performance in a year-over-year comparison to gauge the impact of long-term projects.

Use daily traffic chart to pinpoint the exact day when a traffic increase or decline happened (for instance, due to a Google core update).

Identify pages that account for the biggest traffic losses and improve them. You’ll find this in the Top pages report inside Site Explorer.

4. Conversions

Conversions from organic visits to paid customers are typically hard to measure since this comes down to measuring the ROI of content, which is complicated in itself. However, when we asked marketers about this metric, we found a few interesting ways to solve that problem. For your inspiration, here’s what they measure:

• Conversion as revenue/signups correlation with traffic. This metric assumes that more website visitors increase your chances of turning them into subscribers or buyers.
• Conversion growth from bottom-funnel content. Content aimed at users who are on the brink of purchasing can greatly boost sales because it provides that last bit of persuasion they need to complete a purchase.
• Conversion from first page to paying customer. If the first page a visitor lands on leads to a sale, it’s a clear sign that your content is doing its job effectively.

How to track conversions

For example, in GA4, conversions are called “key events” and are based on tracking user interaction. If a specific event takes place, such as a purchase, a file download, or a form completion, the tool records this as a conversion.

To set up conversion tracking in GA4 you first need to create an event that will be counted as conversion and mark it a key event in the Admin panel of your site (aka property).

Then, to see conversion from the organic traffic channel (the channel you’re optimizing with SEO), go to the Advertising panel.

Here are a few ideas to use this report:

• See how many and which key events were driven by organic search in the last month or quarter.
• See how organic traffic stacks up to other acquisition channels.
• See the share of organic traffic for events with longer conversion paths (the attribution paths tab).

For more information about how to properly set up GA4 for conversion tracking, see this guide.

5. Referring domain growth

Referring domains are essentially the individual websites that link back to your website. By monitoring these, you get a clear picture of how your link profile is expanding over time.

As your link profile grows with more quality links from diverse domains, it helps to build your site’s authority. This authority is crucial because search engines use it as one of the key factors to determine where your pages should rank in search results.

Essentially, the more authoritative your site becomes, the higher your pages are likely to rank and the harder it becomes for others to outrank you.

How to track referring domain growth

• Set up a project (if you haven’t done so yet) and go to your Dashboard.
• Click on the Backlinks card, which gives you a quick insight into backlinks growth.
• Click on the card to get more data (if you need it).

Aim to build as many or ideally more links from unique domains than your competitors to increase your chances for ranking. Read our link-building guide to learn how:

6. Technical SEO issues

• Indexability issues. This means search engines might have trouble finding and adding your website’s pages to their index. If they can’t find your pages, people searching online won’t either.
• Broken pages. These are pages on your website that don’t load properly and might show errors instead (the 404 and 5XX error pagers we all see every now and then). It’s like trying to open a door that won’t budge.
• Few or no internal links. Internal links are connections between one page on your website and another. If your site doesn’t have many, it’s like having a map with missing paths, making it hard for both users and search engines to navigate.
• Mobile experience issues. This refers to problems when your website is accessed on mobile devices. If your site is hard to use on a phone or tablet, visitors might give up and go elsewhere.
• HTTPS issues. HTTPS is a security measure for websites. Ideally, Google wants to show only secure websites.
• Performance and stability issues. This is about how fast your website loads and how smoothly it runs. Typically measured through Core Web Vitals.
• Lack of relevant schema markup. Schema markup is code that helps search engines understand the content on your pages and display it attractively in search results. Without it, your site might not look as inviting or clear when it appears in search and it might not rank in these special spots that Google holds for pages with applied schema.
• Duplicate content. This is when the same or very similar content appears in multiple places on your website. Google will likely show only one of those pages and it might not be the page you want.

Besides these issues, there are more than 100 other possible issues related to less important technical SEO factors and on-page SEO. I won’t cover all of them here since you can learn what they are and how to fix them right inside Ahrefs.

How to track technical SEO issues (aka SEO health)

Use Ahrefs’ Site Audit (free in Ahrefs Webmaster Tools) to monitor for serious technical issues, marked in the tool as “errors”.

• Open Site Audit tool inside Ahrefs.

• Click on Errors in the “Issues distribution” card.

• Go to the issue list, then click on the question mark next to the error and follow the instructions.

To keep your site in good SEO health, schedule regular crawls in Site Audit and fix the most pressing issues.

How to track competitors’ keyword rankings

To track your competitors’ rankings, use a rank tracking tool that allows you to automatically monitor their positions on the keywords you target yourself. So whenever you add keywords you want to target in your strategy, the tool will track both your and your competitors’ rank for that keyword.

In Ahrefs’ Rank Tracker all you need to do is set add your competitors’ URLs (you can track entire domains or specific directories). You can do it as soon as setting up your project or add them later on in the Competitors section.

You can use competitor ranking data to:

• Improve the pages where your competitors outrank you to gain more SOV.
• Set goals and benchmarks.
• Compare historical rankings to your performance over time.
• Quickly see the competitive landscape; see how well you’re doing compared to competitors.
• See how much more traffic you could gain if you outranked competitors.

How to track multiple metrics for a portfolio of pages

You can also track more than just rankings. Using the Portfolios feature in Ahrefs, you can monitor key metrics such as traffic growth and the increase in referring domains for multiple competitors all at once to analyze their overall SEO performance.

You can use this feature to monitor specific pages on your competitors’ sites (such as topics on a blog) or combine all your competitors’ sites to see how your entire niche performs in organic search.

To create a portfolio in Ahrefs, go to the Dashboard and click New > Portfolio, then fill in the URLs you want to track.

Tip

This feature is especially useful if you’re managing SEO for multiple clients — you can track their entire portfolio as one.

It’s also handy if you have multiple authors on your content team; for example, you can track all articles written by a particular author or keep tabs on all guest and freelance posts.

How to track competitors’ new keywords, backlinks, and web mentions

The final method of tracking your competitors allows you to get email alerts when a competitor:

• Ranks for a new keyword. Useful for getting content ideas from your competitors’ new content.
• Rise and fall in keyword rankings. For example, if you see an important keyword suddenly climbing into the top 3, that means your competitor is doing something right, and it’s worth investigating. It’s worth noting that this feature scans, all of the keywords, the site ranks for and not only the ones you track, so it gives you a much wider scope.
• Gain or lose backlinks. Both situations are potential link building opportunities.
• Their brand or product is mentioned online. So, when a competitor gets featured in a review, ranking, or digital PR, you can add that site to your list of link building/PR prospects.

To set it up:

1. Go to Ahrefs Alerts (in the More dropdown menu)
2. Choose the type of Alert you want to set up.
3. Click New alert or choose from one of the projects and fill out the details. In case of the mentions alerts, see our documentation to take advantage of advanced queries.

If you’re doing SEO for someone else, at some point, you will need to put all of those metrics in a report.

In some cases, it may be enough to show the raw data with a few sentences of commentary. This is true in in-house environments when you’re reporting to someone who can interpret the data themselves, especially if you’ve worked with them for a long time.

But if you’re reporting for a client, raw numbers won’t be enough. Additionally, you will need at least these three elements:

• Executive summary: Summarizes the entire report, focusing on major points and outcomes for quick reading by senior stakeholders.
• Opportunities for improvement: Identifies potential areas for SEO enhancements.
• Roadmap: Outlines past achievements and future steps in the SEO strategy.

It’s also important how to report data for your and your stakeholders understanding and convenience. For instance, many clients require a live interactive dashboard with all the data available at all times (similar to these Ahrefs templates for Looker Studio).

Others prefer a document where everything is laid out in layman’s terms — they appreciate the data but they don’t really want to deal with it.

We’ve put together some resources, including a template, to help you quickly and efficiently create a solid report:

Final thoughts

• Set up a routine to check performance. In SEO, consistency is crucial. Conduct weekly checks to assess immediate impacts, like how quickly Google recognizes recent content updates. For long-term planning, such as strategizing quarterly content, review your SEO performance monthly or quarterly. This routine ensures you’re always informed and ready to adapt your strategies as needed.
• Don’t neglect competitor tracking. Failing to track your competitors’ actions could make it challenging to reverse-engineer their strategies later, potentially putting you at a disadvantage.
• Learn about the data before choosing your tools. The reliability and depth of the data provided by an SEO tool are crucial for making accurate assessments and decisions. Before committing to an SEO tool, research its data sources, update frequency, historical data accuracy, and the breadth of its index. You can read about Ahref’s data here and in data studies like this one on search data accuracy.

Got questions or comments? Let me know on X or LinkedIn.

AI can make your SEO efforts faster, better, and more fun—if you know how to use it.

Here are 14 practical ways to get faster, more efficient SEO results with help from your robot overlords friends.

• When people talk about “AI” today, they generally mean generative AI: software models that can create text or images. ChatGPT is the most popular tool.
• Generative AI works like a super-powerful autocomplete. It has “read” billions of documents and is very good at creating similar documents in response to your prompts.
• Generative AI is very good at summarizing information, short-form writing, and sounding confident. It’s bad at math, consistency, and anything that involves understanding the “bigger picture” (like writing an SEO strategy for your particular company).
• AI generally doesn’t tell you when it’s misunderstood your question, can’t do what you’ve asked, or doesn’t have the right data available—so treat all of its answers with healthy skepticism.

With those ideas lodged in your brain, let’s look at how you can use AI tools for faster, better SEO.

Suggest seed keywords

“Seed” keywords are words and phrases related to your business that you can use as the starting point for keyword research.

Pick a starting topic and ask AI to suggest related keywords: sub-topics, questions, similar concepts, you name it.

Take your list of ideas, plug them into a keyword research tool like Ahrefs’ Keywords Explorer, and you can quickly see the estimated traffic potential and Keyword Difficulty for each of these terms:

Not all of these seed keywords will have meaningful volume, but that’s okay. Switch to the Matching terms or Related terms tabs, and you’ll see hundreds more related keywords that do:

You can even skip the ChatGPT part entirely and use the built-in AI suggestion feature in Keywords Explorer:

Here, our AI copilot has brainstormed “subtopics and niche areas” related to our core topic, content strategy:

Analyze SERP intent

AI can help you understand the different types of search intent present in a particular SERP (search engine results page).

This can be useful for working out which type of content you need to create to match the dominant intent (do searchers want an informational guide, or a free tool?).

In the example below, I copy/pasted page titles from the SERP for the keyword “LLM” and asked ChatGPT to categorize them by the different intent types present:

After a little cajoling and refining, ChatGPT grouped the titles into a few different categories, like definitional (explaining what an LLM is) and comparative (comparing different types of AI models):

You can take this process to the next level with the Identify intents feature in Ahrefs. For your given keyword, scroll to the SERP overview report, and hit the “Identify intents” button:

This has the benefit of also showing you the percentage of total estimated traffic each intent receives.

In this example, with 82% of traffic, it makes sense to target the keyword “llm” with a definitional article about LLMs, and ignore the lower traffic intent associated with LLM degree programs.

AI can be used to pump out complete articles, but you’ll get better results—and have less risk of a Google penalty—if you use it like a creative sparring partner for your content creation process.

Brainstorm titles and headers

Titles and headers have a crucial indirect role in SEO by encouraging readers to actually click and read through your content. AI can dramatically speed up the process of brainstorming titles and headers.

Here, I’ve pasted the content of my latest blog post into ChatGPT and asked it to suggest title ideas:

I generally don’t use these ideas verbatim, but ChatGPT regularly generates words or phrases that make their way into my finished titles.

You can also use our free blog post title generator in the same way. Just describe your topic, choose the writing tone, and hit “generate”:

You can modify and create new ideas at the click of a button:

Check grammar

AI is great for checking writing for grammar mistakes. Here, I’ve pasted an article paragraph into our free AI grammar checker…

…and a moment later, AI has flagged two possible issues for me to resolve:

Edit transcripts

Maybe you’ve interviewed an expert and want to add their quotes and experience into your search content. Or maybe your team has created a YouTube video that you’d like to repurpose into a keyword-targeted article.

In either case, you can use AI to tidy up and correctly format transcripts, making it much easier to pull out quotes and ideas.

In this example, I’ve asked ChatGPT to correct a free (and error-prone) transcript from a YouTube video:

And here’s the edited version, complete with correctly capitalized brand names, removed typos, and grammatically correct punctuation:

SEO is a never-ending process, and AI can be a useful tool for speeding up some of the ongoing optimization tasks you’ll need to make to keep your pages ranking.

Add missing topics

Here’s an example for the keyword programmatic seo, comparing the content of my article to the content of other top-ranking pieces. We can immediately see a couple of missing topic areas:

Content Grader can even explain how you should address the topic gap, and share an example from another top-ranking article:

Write meta descriptions

Good meta descriptions encourage searchers to click on your pages, but Google has a tendency to change and rewrite even the most carefully-crafted meta descriptions.

If you want to generate lots of meta descriptions without sinking tons of time into the process, AI is pretty perfect. Here’s our free AI meta description generator: just describe the contents of your page, choose a writing tone and the number of variations you’d like, and hit generate.

And here are the outputs:

Make content more helpful

Aleyda Solis created a custom GPT (a specially trained AI model) that reviews content according to Google’s helpful content guidelines.

While I don’t think it’s a replacement for the skilled judgment of a professional SEO, it can offer a quick, automated way to pinpoint obvious problems with content.

Here I’ve asked it to compare my article on programmatic SEO to a competing article:

It’s easy to mess up certain parts of technical SEO, like schema or hreflang implementation. From my experience, AI is better and more reliable than I am in these areas.

Create schema markup

Adding schema markup to relevant content types (like recipes or reviews) can help your pages become eligible for Rich Results, special Google features that include a ton of extra data about your content.

Here, I’ve asked for recipe schema for a chicken soup recipe. With a couple of tweaks (like adding in the recipe author), I could add this to my page and become eligible for rich results (and most likely more clicks):

Generate hreflang

Hreflang is an HTML attribute that tells search engines about the multiple versions of a page for different languages or regions. Here, ChatGPT has written the hreflang tags for four different versions of my blog post:

AI is great at helping with these analytical and reporting tasks, from digging through performance data to see which tactics work, to sharing your findings in easy-to-communicate ways with your company or clients.

Of all the AI/SEO use cases I’ve covered, these are probably my favorite.

Constructing regex queries

Here’s ChatGPT helping me extract URLs from a list of email addresses, combining regex queries with a Google Sheets formula:

And here it’s helping me filter a spreadsheet of URLs by their crawl depth:

And here it’s written a query to use with Ahrefs Site Audit to help me filter out localized content (pages that have country codes, like /de/ for Germany, somewhere in their URL):

Making Google Sheets formula

Here I’ve described the structure of an article reporting spreadsheet to ChatGPT, and asked for a very complicated formula to allow me to filter for certain types of published articles. It doesn’t even break a sweat:

It’s also great for troubleshooting when things go wrong:

Writing Python scripts

Python is a popular language for automating SEO processes. Generative AI is pretty darn handy at writing and troubleshooting Python code, and I’ve used it to help speed up some of my SEO processes.

Here, I asked AI to create a basic web scraper for storing data from a given webpage:

And here I asked for help writing a script to call the Ahrefs API and collect bulk traffic and backlink data for a list of websites:

And yes—both of these scripts worked!

Vizualize performance data

All of the visuals in this section were created with ChatGPT, Ahrefs data, and a little know-how.

For longer explanations (and the prompts used to make these visualizations), check out Patrick’s article:

Here’s a graph of organic traffic over time, with traffic anomalies (usually Google updates) highlighted:

Here’s a plot comparing desktop and mobile rankings for a selection of keywords:

And here’s a chart showing seasonal patterns in backlink acquisition:

Does Google penalize AI content?

No, not strictly speaking. Google penalizes bad content, and AI makes it easy to make bad content.

Some companies use AI to dramatically scale and automate their content creation. When this content is thin, there’s a chance that Google will issue a manual spam penalty. In this example, a site used AI to publish 1,800 thin articles and received a penalty, tanking their traffic to virtually zero:

As I’ve written before,

“I don’t think that publishing AI content means an automatic penalty. AI content detectors don’t work, and even if they did, Google is apparently agnostic to AI use—but it is not agnostic to bad content or bad actors. And AI makes it very easy to make bad content.”

It’s a good idea to use AI to improve the efficiency or quality of your content, but not to pump out thin spam content.

Is Google losing market share to AI?

Google has always been the main search engine SEOs care about, and in the age of AI… that hasn’t really changed. According to Statcounter, Google’s market share has held relatively steady at a staggering 91%:

But although Google’s dominance over the search market is pretty unchallenged, there are more alternatives than ever. These are useful for seeing where Google might take inspiration and improve its own search experience in the future:

• Competing search engines are offering their own AI features (like Bing or our Yep.com).
• Companies like Perplexity.ai offer an alternative search experience built entirely on AI models
• Some people are even building their own AI chatbots trained on specific bodies of work—instead of asking Google for health and fitness advice, you could ask a chatbot trained on the Huberman Labs podcast.

Will SGE reduce traffic from certain keywords?

Google has just launched AI Overviews (formerly known as Search Generative Experience, or SGE). AI Overviews seem to work a lot like featured snippets: they try to answer the searcher’s query directly, right there in the SERP, without the need to click on another website.

There’s a concern that many websites will see a decline in search traffic from AI Overviews, and some SEOs even suggest trying to optimize your content for AI Overviews.

While we wait to see what impact AI Overviews has on traffic from Google Search, the best response is to focus on topics that can’t be neatly summarized in a single paragraph.

We call these “deep topics”: areas where AI can’t provide everything the reader needs, because there are lots of possible answers, or it requires firsthand experience.

Does Google reward first-person experience?

Google already has a plan for stopping SERPs from being swamped by copycat AI content, and it involves prioritizing content that includes EEAT: expertise, experience, authority, and trust:

 

“There are some situations where really what you value most is content produced by someone who has first-hand, life experience on the topic at hand.”

EEAT is used by Google’s Quality Raters, whose experiences may be used to train Google’s machine learning models to help them identify “quality” content.

But Google aside, EEAT is great for readers, so it’s worth incorporating into your SEO strategy even if you won’t see an immediate ranking boost. There are three simple ways we recommend standing out from AI content:

• Experimentation: create proprietary data.
• Experience: share your real, lived experiences.
• Effort: go further than competing content.

Final thoughts

If you want to test out some AI tools in the easiest possible way, try experimenting with our 40 free AI writing tools. They can help with everything from writing clickable titles to generating tons of meta descriptions, and help you separate AI fact from AI fiction.

A lot is happening in the world of search right now, and for many, keeping pace with these changes can be overwhelming.

The rise of chatbots and AI assistants – like ChatGPT and its new model GPT-4o, along with Google’s rollout of AI Overviews and Search Generative Experience (SGE) – is blurring the lines between chatbots and search engines.

New AI-first entrants, such as Perplexity and You.com, also fragment the search space.

While this causes some confusion and necessitates that marketers pivot and optimize for multiple types of “engines,” it also presents a whole new array of opportunities for SEO pros to optimize for both traditional and AI-driven search engines in a new multisearch universe.

This evolution raises a broader question – perhaps for another day – about redefining what we call SEO to encompass terms like Artificial Intelligence Optimization (AIO) and Generative Engine Optimization (GEO).

Currently, every naming convention seems subject to change, which is something to consider as I write this article.

Either way, this evolution opens up tremendous opportunities for disruption in the overall search landscape.

What Is A Chatbot Or AI Assistant?

At the most basic level, chatbots use natural language processing (NLP) and large language models (LLMs) that are trained to extract data from online information, sources, and specific datasets. They then classify and fine-tune text and visual outputs based on a user’s prompt or question.

Chatbots are often used within specific applications or platforms, such as customer service websites, messaging apps, or ecommerce sites. They are designed to address specific queries or tasks within these defined contexts.

Right now, we see many crossovers between LLM-based chatbots and search engines. Rapid developments in these areas can cause confusion.

In this article, we’ll focus on the development of AI models in chatbots and their relation to search, with an inferred reference between chatbots and AI assistants.

The Evolution Of Chatbots And AI Models

Since ChatGPT emerged in November 2022, we’ve seen a significant boom in chatbots and AI assistants. Now, generative AI allows users to interact directly with AI and engage in human-like conversations to ask questions and complete various tasks.

For example, these AI tools can assist with SEO tasks, create content, compose emails, write essays, and even handle coding and programming tasks.

As they evolve, chatbots become multimodal (MMLLMs), improving capabilities beyond text to include images, audio, and more.

Image from 2024 AI Index Report from Stanford University, May 2024

For those interested in digging deeper into these models, the 2024 AI Index Report from Stanford University is a great resource for SEJ readers.

While many chatbots and AI models serve similar purposes, they also have distinct applications and use cases, such as content creation, image generation, and voice recognition.

Here are a few examples with some interesting differentiators and points:

• ChatGPT: Conversational AI for research, ideation, text, image content, and more.
• Google Gemini and Gemma: Uses Google’s LLM to connect and find sources within Google.
• Microsoft Bing: Uses ChatGPT for conversational web search in Bing.
• Anthropic Claude: Various AI models for content generation, images, and coding.
• Stability AI: Suite of models and AI assistants for text, image, audio, and coding.
• Meta Llama3: Utilizes Facebook’s social graph, its own Llama 3 model, and real-time data from Google.
• Microsoft’s Copilot: AI assistant for business creativity and productivity apps.
• Amazon LLM and Codewhisperer: Enhances customer and employer experiences.
• Perplexity AI: Provides quick answers, sources of information, and citations.

Perplexity AI (which I will touch on later in this article) acts more like a search engine than many other chatbots and AI assistants.

Beyond their primary use cases, many companies are making their models available to a wider audience and broader ecosystems, allowing users to customize their own AI assistants.

For example, Amazon’s Bedrock enables AWS customers to use Anthropic and other LLMs, including Amazon’s own model, to create custom AI agents. Companies like Lonely Planet, Coda, and United Airlines are already using it.

On May 13, OpenAI launched its new flagship model, GPT-4. This model is a combination of AI technologies, bringing together what OpenAI calls “text, vision, and audio.” It also opens up access to its application programming interface (API), allowing developers to build their own applications.

All of this convergence has a lot of people wondering.

What’s The Difference Between Chatbots And Search Engines?

The first thing to note is that both chatbots and search engines are designed to provide information.

Search engines and some chatbot models share many similarities, which means their definitions can blur, and the relationships between them converge and collide.

However, at the moment (but it is changing), there is still a distinct difference between the two:

Search Engines

• Search engines are better for exploring a wide range of topics.
• They provide diverse perspectives from multiple sources.

Chatbots

• Chatbots are better for quick answers, task completion, and personalized interactions.
• They enhance the efficiency of the average searcher, making them much more effective at finding information.

Image from author, May 2024

As more overlays and overlaps occur, the definitions of what constitutes a chatbot, an AI assistant, and a search engine may need to be redefined.

How Chatbots And Search Engines Work Together

Conversational search is a key area where search engines increasingly integrate chatbot features to provide a more interactive search experience.

You can ask questions in natural language, and the search engine may respond with direct answers or engage in a dialogue to refine your query.

Chatbots and AI assistants often utilize search engine technology to access information from the web, enhancing their ability to provide accurate and comprehensive answers.

This integration allows chatbots to go beyond their programmed knowledge base and tap into a broader range of information.

Here are a few examples:

• Google: Integrates its own chatbot features into its search engine through SGE, providing direct answers and engaging in conversational search for some queries.
• Bing: Incorporates a chatbot called “Bing Chat” that uses ChatGPT, conversational AI, and search technology to answer questions and provide information.
• YouChat: A search engine that provides conversational responses to queries and allows for follow-up questions.
• Meta: Utilizes its social graph and Google’s real-time data in its chatbot/AI assistant.
• Perplexity AI: A chatbot that functions like a search engine, focusing on informational sources, sites, and citations.

These examples illustrate how the lines between chatbots and search engines are blurring. Thousands more instances show this convergence, highlighting the evolving landscape of digital search and AI.

How “Traditional” Search Engines Are Evolving As AI-First Entrants Arrive

The rise of generative AI and chatbots has caused significant upheaval in the traditional search space.

Traditional search engines are evolving into “answer engines.” This transformation is driven by the need to provide users with direct, conversational responses rather than just a list of links.

The line between chatbot engines and AI-led search engines is becoming increasingly blurred.

While AI in search is not a new concept, the introduction of generative AI and chatbots has necessitated a seismic shift in how search engines operate. For the first time, users can interact with AI in a conversational way, prompting giants like Google and Microsoft to adapt.

On May 14 at Google IO, Google announced the roll-out of AI Overviews as it integrates AI features into its search engine. It is also making upgrades to SGE.

The ultimate goal is to enhance its ability to provide direct answers and engage in conversational search. This evolution signifies Google’s commitment to maintaining its leadership in the search space by leveraging AI to meet user expectations.

In a recent interview on Wired Magazine titled It’s the End of Google Search As We Know It, Google Head of Search, Liz Reid, was clear that:

“AI Overviews like this won’t show up for every search result, even if the feature is now becoming more prevalent.”

As my co-founder, Jim Yu, states in the same article:

“The paradigm of search for the last 20 years has been that the search engine pulls a lot of information and gives you the links. Now the search engine does all the searches for you and summarizes the results and gives you a formative opinion.”

Beyond Google, we are seeing a rise in new, AI-driven search engines like Perplexity, You.com, and Brave, which act more like traditional search engines by providing informational sources, sites, and citations.

These platforms leverage generative AI to deliver comprehensive answers and facilitate follow-up questions, challenging the dominance of established players.

Meta is also entering the fray by utilizing its social graph and real-time data from Google in its AI assistant, further contributing to the convergence of search and AI technologies.

At the same time, according to Digiday, TikTok is starting to reward what it calls “search value.”

Going forward, it’s important to remember that people have diverse needs, and we turn to different platforms for specific purposes.

Just as we go to Amazon for products, Yelp for restaurant suggestions, and YouTube for videos, the rise of AI will only amplify this trend. Each search engine will find its niche, leveraging its strengths to cater to particular user requirements.

ChatGPT is an intriguing case that stands out not for its research capabilities but for its prowess in content creation. While it excels in crafting high-quality content, its research functionalities fall short.

Effective research relies on real-time data, which platforms like ChatGPT currently lack. As we move forward, we expect to see search engines specialize even further, each excelling in specific areas based on its unique strengths and features.

What Does It All Mean For Marketers?

This fast-moving landscape and the convergence of search and AI presents both challenges and opportunities for marketers.

Optimizing for one engine is no longer sufficient; it’s essential to target multiple platforms – each with unique users, demographics, and intents.

Here’s how marketers can adapt and thrive in this dynamic environment.

Optimizing For Different Platforms

Google

• Strength: Dominates the traditional search space with a vast user base and comprehensive data sources.
• Tip: Focus on core technical SEO, including schema markup and mobile optimization. Google’s Search Generative Experience means direct answers are becoming more prevalent, so structured data and high-quality content are vital.

Perplexity AI

• Strength: Provides detailed citations and emphasizes source material, driving referral traffic back to original sites.
• Tip: Ensure your content is authoritative and well-cited. Being a reliable source will increase the likelihood of your site being referenced, which can drive traffic and enhance brand trust.

ChatGPT

• Strength: Excels in conversational AI, making it suitable for quick answers and personalized interactions.
• Tip: Create engaging, concise content that answers common questions directly. Utilize conversational language in your SEO strategy to match the style of ChatGPT interactions.

Key Strategies For Marketers

From optimizing technical SEO to harnessing the power of semantic understanding and creativity, these strategies provide a roadmap for success in the era of AI-driven search.

Core Technical SEO

Basics like site speed, mobile-friendliness, and proper schema markup remain crucial. Ensuring your site is technically sound helps all search engines index and rank your content effectively.

Semantic Understanding

Search engines and conversational AI are increasingly focused on semantic search. Optimize for natural language queries and long-tail keywords to match user intent more accurately.

Content And Creativity

High-quality, creative content is more important than ever. Unique, valuable content that engages users will stand out in both traditional and AI-driven search results.

Expanded Role Of SEO

SEO now encompasses content creation, branding, public relations, and AIO. Marketers who can adapt to these roles will be more successful in the evolving search landscape.

Be The Source That Gets Cited

Ensure your content is authoritative and well-researched. Being a primary source will increase the likelihood of citations that drive traffic and enhance credibility.

Get Predictive

Anticipate follow-up questions and provide comprehensive answers. This will not only improve user experience but also increase the chances of your content being highlighted in AI-driven search results.

Brand Authority

Focus on areas where your brand excels. AI search engines prioritize authoritative sources, so build and maintain your reputation in key areas to stay competitive.

The Best Content That Provides The Best Experience Wins

Ultimately, the quality of your content will determine your success. Invest in creating the best possible user experience, from engaging visuals to informative text.

Key Takeaways

Today, search encompasses a dual purpose: It can serve as a standalone assistant-based application or integrate into search engines for AI-led conversational experiences.

This fusion presents marketers with a unique opportunity to elevate their brands by creating accurate and authoritative content that positions them as trusted sources in their respective fields.

Ranking on the first page and being recognized as the go-to source cited by AI engines is no less important than 10 or 20 years ago but is exponentially more difficult.

The good news is that whether it’s Google’s AI engine or newcomers like Perplexity, brands that establish themselves as authorities in their niche stand to benefit immensely.

Marketers need to embrace creativity and collaboration across omnichannel teams. Ensure that your website is visible and accessible to all types of engines, whether traditional or AI-driven.

I’d like to leave you with a few questions to consider as you find your way forward in this complex environment. Pardon the pun, but no one has all the right answers yet.

• Are chatbots morphing into search engines?
• How do social platforms differentiate as younger generations look to them as search engines?
• How would you define a search engine?
• Who will win the race for user loyalty – traditional search engines infused with AI or new entrants built on generative AI from the beginning?
• How would you redefine your role as an SEO – are you AI first?

While you consider that, stay proactive and adaptable and position yourself and your company to leverage the diversity and complexity of the search ecosystem to your advantage. In a world of ChatGPT, chatbots, and AI in search, you’re not optimizing for one channel, such as Google or Bing.

Successful optimization in this multifaceted landscape calls for a holistic approach. It’s not about keyword rankings or click-through rates; it’s about unraveling the intricacies of each platform and adjusting your strategies accordingly.

This means optimizing your content for conversational search, tapping into the capabilities of AI to tailor user experiences, and seamlessly integrating across different channels and devices.

Leverage the strengths of each platform to amplify your message by use case and engage with your audience on a deeper level, and you’ll ultimately drive more meaningful results for your business.

More resources: 

Featured Image: Memory Stockphoto/Shutterstock