Sitemap. While this is a term you may be familiar with, what does it mean?

Do you need one? Where do you find one? How do you make one?

These are valid questions; for some, there might be more than one answer.

Today, we will take a deep dive into the sitemap world, so that you can walk away with the necessary answers and confidence around the topic!

What Is A Sitemap?

Let’s start here.

Defining a sitemap is essential for several reasons, and we are going to go through the two main types that apply to technical SEO: XML and HTML sitemaps.

XML Sitemap

An XML sitemap is a file that provides a website’s essential pages, videos, and other important files for Google to discover when crawling the site.

Not only are these listed in the file, but the sitemap can also provide details for Google to know – for instance, when the page was last updated, and if the content is available in other languages.

As I mentioned, you can also provide details about content types like videos, photos, and news-related content, specifically in your XML sitemap.

According to the Google Developers Sitemaps section, the following can be included for specific types of content in your sitemap:

• A sitemap video entry can specify the video running time, rating, and age-appropriateness rating.
• A sitemap image entry can include the location of the images included on a page.
• A sitemap news entry can include the article title and publication date.

Next, we will talk about what an HTML sitemap is and the differences between the two.

HTML Sitemap

An HTML sitemap is more targeted for users on your site than for Google.

This is a page that exists on your site and has links to the pages on your website – and in some cases, includes a little context into what those pages are.

Google mentions that you should try to establish a consistent and clear hierarchy on the HTML sitemap as, although not its purpose, it can help with indexation.

You can think of an HTML sitemap as a directory that users can leverage to navigate your site and find what they need.

An HTML sitemap should not be an attempt to replace the important pages in your site’s navigation.

XML Sitemaps Vs. HTML Sitemaps

So, what are the key differences between these two types of sitemaps? Let’s review.

XML

• The intent is for Google and other bots.
• There is no hierarchy.
• Used primarily for indexing.
• You can submit via Google Webmaster Tools.

HTML

• The intent is for users.
• A hierarchy should be used.
• No place to submit in Google Webmaster Tools.

Do You Need A Sitemap?

If you are wondering if you need a sitemap, that depends!

First, let’s discuss the XML sitemap. There are a few questions you can ask to determine if you need an XML sitemap:

• How big is your site? Is it large enough that Google may miss newly updated content when it is crawling?
• Is your site relatively new? If so, it may not have a ton of external links on the Internet that point to it to help Google discover it. Even if your site isn’t new, and you don’t have external links, your answer to this question should be yes.
• Is your site content heavy? Do you have many photos, videos, news content, etc.?
• Does your site need a better architecture that results in pages not being well linked to each other? This can also be the case with archived and orphan pages you want to be indexed.

If you answered yes to any of the questions above, then yes, it is best practice to have an XML sitemap.

Even if you answered no to all of the above, I would recommend an XML sitemap for a few reasons; If your site grows, expands its scope, and other situations may arise, having a sitemap will be beneficial!

Next, let’s review whether it makes sense for you to have an HTML sitemap.  Depending on where you look, you will find that answer to be yes or no.

HTML sitemaps are known to be an older concept, but that doesn’t mean you shouldn’t have one.

The XML sitemap has the information needed for Google to crawl, index, and learn other important information about these pages. However, an XML sitemap does not show hierarchy like an HTML sitemap.

Google will crawl the links on your site, and including an HTML sitemap could allow Google to understand your site’s architecture and relationships better.

This is even more useful for sites that have an incredibly large number of pages.

So, is having an HTML sitemap critical? No, it is not.

It is also not a cure-all for a poorly architected and nested website. While it isn’t a critical element of success, it has shown benefits that make having one a best practice.

To close this topic out, I recommend you have an XML and HTML sitemap because let’s be honest, why not, when the pros outweigh the cons very clearly?

Now you may be wondering how to create these two assets and what to do with them – so, let’s jump into some ways you can create these files and where to put them on the site.

How To Create An XML Sitemap

First, we will go over how you can generate sitemaps from scratch, and then we will get into some great tools that can do it for you.

XML sitemaps have specific criteria in order to be rendered valid.

Below are a few specific requirements for XML sitemaps:

• Begin with a <urlset> tag and end with that tag closing </urlset>.
• Include the protocol you are using within the <urlset> tag.
• Each URL entry must have a <url> tag as a parent XML tag.
• Include a <loc> child entry for each <url> parent tag.
• Each sitemap can only contain up to 50,000 URLs and 50MB.
• Must be UTF-encoded.

XML Sitemap Best Practices

Now, let’s look at some key best practices when it comes to creating XML sitemaps:

• Only URLs you want to be indexed should appear in your sitemap. This means no redirected URLs, non-canonical URLs, or pages marked as no-index.
• Do not use session Ids.
• Only include the primary if you have two versions (mobile and desktop) of your site.
• Include media assets like videos, photos, and news items.
• Use hreflang to show Google that there are alternative language versions of your website.
• Google documentation notes it leverages <lastmod>, but only if it’s consistent and verifiable. If you can’t keep this accurate, don’t use it.
• Google ignores the <priority> and <changefreq> tags at this time, according to John Mueller on this Search Off the Record podcast.
• Google will not crawl your URLs in the order they are listed, nor does it guarantee indexation.
• Your sitemap should be updated regularly – automatically, or manually – or Google may not trust it.

Now, if you felt lost reading those beginning requirements, that is okay, because there are tools to help you achieve your desired outcomes! We will go over some later in this article.

Check out the refined version below:

Screenshot from lowes.com, January 2023

How To Create An HTML Sitemap

When putting together an HTML sitemap, remember its purpose is to serve a user on the site and help Google understand the hierarchy of your website.

You do not want to no index this page from Google; keep it crawlable!

You will want to ensure you don’t just throw thousands of links on an HTML sitemap page with no sense of organization, as this won’t help anyone – bots included.

Screenshot from Home Depot, January 2023

HTML Sitemap Best Practices

Let’s go over a few quick best practices when it comes to HTML sitemaps:

• Arrange the page’s structure to align with your website’s structure. You will want to make sure that the hierarchy is easily understood.
• The HTML sitemap should be located somewhere the user can easily find it. You will often see it in the footer links of a website.
• Use anchor text that is valuable to the user.

Need a little help getting started? No worries – there are plenty of tools to help you.

Sitemap Generator Tools

There are a number of tools to help you generate different types of sitemaps. Let’s go over a few now.

XML Sitemap Generator Tools

• Screaming Frog – This tool is a great option for generating a sitemap, especially if you want to generate one after crawling your URLs. Screaming Frog is free if you have under 1,000 URLs, but you would have to buy a license if you have more.
• XML-Sitemaps.com – This web-based application allows you to enter your website URL and it generates an XML file for you. This is a free tool for up to 500 URLs.

Depending on which CMS you are leveraging, there are also thousands of XML sitemap generator plug-ins, but be cautious as even the best generator tools have their limitations, so make sure to double-check the output.

Here are a few popular XML sitemap plugins for WordPress:

HTML Sitemap Generator Tools

• com: this is a free online tool where you can scan your website URL or upload a document to generate an HTML sitemap. As we discussed earlier, there may be better approaches than a generator if your site is poorly architected.
• Crawler: Like Eli mentions, if you have a large site and are already using a crawler like OnCrawl, DeepCrawl, Screaming Frog, or SiteBulb, you can leverage the output from a crawl to help generate your HTML sitemap.

Like XML sitemaps, there are also a variety of CMS plugins for creating HTML sitemaps. Here are a few for WordPress:

In Conclusion

Sitemaps have existed in the SEO world for some time as a method for helping search engines discover and crawl websites.

And, while having a sitemap isn’t always necessary for every site, it certainly doesn’t hurt – and can be especially useful for both new and large sites.

When you are determining your next steps for creating a sitemap for your website – whether XML or HTML – I hope you can leverage this guide to decide which path makes the most sense for your site’s needs.

More resources: 

Featured Image: Sammby/Shutterstock

Deciding whether a keyword should be targeted by a separate page or clustered with other keywords is a common problem in SEO. Keyword mapping is a process aimed at solving this.

Keyword mapping is popularly defined as assigning keywords to pages. But what you really need to solve the problem is assigning topics to content types. 

In this article, I’ll explain the benefits of this approach and, more importantly, I’ll show you the process. No templates required.

Fact 1. Google may see seemingly different keywords as the same topic.

For example, we rank for these keywords in the top 10 with a single page: 

• “seo basics”
• “how to use seo” 
• “beginner’s guide to seo”
• “getting started with seo”
• “seo knowledge”

Fact 2. Conversely, Google may see seemingly similar keywords as different topics. 

For example, let’s compare “digital marketing” with “online marketing.” I’d say those two keywords are pretty close to each other. Google disagrees. 

The above two facts are also reasons why keyword mapping by just relying on keywords is not the optimal way. You won’t know whether you’re wasting your time targeting the same topic with different keywords or just “confusing” Google. 

But why content types instead of pages or even URLs? Because before you decide what page will be used to target the keyword, you’ll need to identify the search intent of the keyword. And a good starting point for that is identifying the dominating type of content on the first page of Google. 

To sum up, the benefits of keyword mapping using topics and content types are: 

• Seeing keywords the same way Google sees them: as topics and subtopics. 
• Incorporating search intent into the process. 
• Keeping an organized list of topics, which also helps to prevent duplicating content.

Going further, we’ll look at two levels of using this method: the fast lane and the more thorough one. 

Learn more: What Is Semantic Search? How It Impacts SEO 

You’ll need a keyword research tool that can do keyword grouping based on what’s on the SERP, such as Ahrefs’ Keywords Explorer. In the case of this tool: 

1. Enter your keywords
2. Open Matching terms report
3. Go to the Parent topics tab 

If you click on a Parent Topic, you will find separate topics “distilled” from your keywords. So for example, you will see keywords like “can babies get covid” and “babies and covid” grouped under the same topic. 

At this level of keyword mapping, your target keyword is the Parent Topic (not the keywords inside that Parent Topic). 

The next step is to identify the content type. The easiest way to do this is to see what kind of content dominates the first three to five results in Google. 

Typical content types are:

• Articles
• Videos
• Product pages
• Product category pages
• Landing pages 

As a result, assigning topics to content types will give you a super simple yet highly actionable database.

So this is the fast method. The great thing about it is that it automates keyword grouping by using real SERP data (and not just semantics). 

However, it has its downsides too. Sometimes, it “hides” less popular topics that could potentially be targeted with a separate page. Here’s why. 

The parent keyword is derived from the top-ranking page on the SERP. If Google thinks that the best answer to the query is found on a page that is targeting a broader topic, it will still use it. This may result in a confusing SERP like this one: 

This kind of situation probably won’t happen too often. But if you want to squeeze everything out of your keyword mapping process, you need to go to level 2. 

In level 2, we’re going to take a closer look at the Parent Topics to see what’s in them. 

1. First, you should pick a Parent Topic.
2. Sort keywords inside the topic by KD (Keyword Difficulty). Big differences in KD will be an indication of a different set of pages on the SERP.
3. If you see a keyword with a significantly different KD than the Parent Topic, click on the SERP button.
4. See if the top-ranking pages, excluding the first result, talk about the keyword instead of the Parent Topic. You can use the Compare with feature for a quick overview of the situation. The lower the SERP similarity score, the higher the probability you’re looking at two different topics. 

Let’s look at a couple of examples. 

In the first example, we’ve got a keyword with a KD score that’s 20 higher than the Parent Topic. Upon investigating, we see that we may be dealing with two separate topics: The SERP similarity is quite low. Also, there is only one common result, while other pages target the keyword directly. 

Next example. Here we have “teething symptoms” (KD 65) and “when do babies get molars” (KD 28). Looking at SERP similarity, we see that this, again, may be a case of two topics. 

But there’s more. Only the bottom results target the keyword directly. Others talk about teething timelines, stages, charts, etc. This is a hint for yet another way to rank for the keyword. 

Generally speaking, when you see that you’re dealing with a separate topic “in disguise,” the decision comes down to:

1. Targeting the Parent Topic anyway. For example, if the top result is a featured snippet, you may be able to win it with a page on a relevant broader topic. 
2. Marking the keyword as a separate topic and targeting it directly with a separate page. In this case, add that keyword as a topic to target and note down the content type. 
3. Turning to SERP analysis in tougher cases (like our example above). 

Final thoughts 

Feel free to customize the process and add your own data points. If you feel like going a step further and assigning URLs, your website folders, or introducing some kind of prioritization (e.g., business potential), this won’t hurt. 

However, keep in mind that keyword mapping is not a good way to design your entire website structure. Most often than not, not all pages on your site should be search-based. 

What are the next steps after keyword mapping? 

Got comments or questions? Ping me on Twitter or Mastodon. 

Of all the many, many functions available in Google Ads, I have a few that are my favorites. And sitelink assets – previously known as sitelink extensions – are at the top of my list.

Why? Because they’re so versatile. You can do almost anything with them if you think through your strategy carefully.

For example, you can use the mighty sitelink in your advertising to:

• Promote low search volume themes.
• Push lagging products out the door.
• Maximize hot sellers.
• Highlight certain product categories.
• Answer common questions.
• Handle PR problems.

And that’s just a start! Sitelink assets can almost do it all.

Best Practices For Using Sitelink Assets Extensions

If you truly want to get the most out of your sitelinks, you need to think about your intention.

To help you with that, I’m going to lay out a few sitelink guidelines.

1. Get clear on your objectives. Before you start, you need to think about your goals. What are you trying to achieve with these assets? Are you advertising products or services? Will the asset work well with both branded and non-branded keywords? Your answers to these questions will help determine if your sitelinks are versatile and useful to the searcher.

2. Use sitelinks as part of your larger strategy. Don’t think of your sitelinks in isolation. You should also consider the accompanying ad, landing page, and other assets. Make sure they all work together in service to your overarching strategy.

3. Use a mix of sitelinks. Sitelinks can serve multiple purposes, so make sure you’re using a variety. For example, you don’t want to use every sitelink on an ad to promote on-sale products. Instead, use a mix. One could promote an on-sale product, one could generate leads, one could highlight a new product category, and one could direct prospective clients to useful information.

4. Create landing pages for your sitelinks. Ideally, you want to send users to landing pages that tightly correlate with your sitelink instead of just a regular page on your website.

5. Track sitelink performance and adjust. It’s not enough to set up sitelinks. You should also track them to see which links are getting traction and which ones are not. This doesn’t mean that all sitelinks should perform equally (more on this below), but it does mean they should perform well given their type and objectives.

Why it’s Better To Use A Mix Of Sitelink Assets

Let’s dive deeper into this idea of using a mix of sitelinks by looking at an example.

In a new client account, we created four different types of sitelinks:

• Two sitelinks are product-focused (as requested by the client).
• One sitelink connects users with an engineer to learn more about the product (“Speak to an Engineer”). It has more of a sales focus.
• One sitelink allows users to learn more about the products without speaking to an engineer (“What is?”).

The “What is?” sitelink is outperforming the “Speak to an Engineer” sitelink when we measure by CTR. While we need more data before making any changes, I predict we’ll eventually swap out the sales-y “Speak to an Engineer” sitelink for something else.

The fact that the educational link (“What is?”) is performing better than the sales-y link (“Speak to an Engineer”) isn’t too surprising in this case. The product is a new, cutting-edge robot that not many people are aware of, yet. They want more info before talking to someone.

By using a mix of sitelinks, and assessing the performance of each, we gained a lot of valuable information that is helping to guide our strategy for this account. So going with a mix of sitelinks is always a good idea. You never know what you’ll discover!

Sitelink Assets Examples

Now, let’s look at some specific examples of sitelink assets in Google Ads.

Example 1: Chromatography

Screenshot from Google, January 2023

Application Search: This ad is for a highly technical product that can be used in a wide variety of applications. (Chromatography is a laboratory technique for separating mixtures.) So putting “application search” in a sitelink here might make sense. It helps prospective clients find what they’re looking for.

Sign up and Save Big: A good sitelink for lead generation and potential revenue.

Technical Support: I’m not a big fan of putting technical support in sitelinks. Tech support seems more targeted to current users rather than prospective users. But who knows, maybe they really do want to help current users get tech support via their advertising.

Guides and Posters: Again, this sitelink is a bit unusual, but it might be appropriate for this product. Perhaps people are downloading branded posters and posting them in their workplaces. If so, it’s a great way to build brand awareness.

Example 2: Neuroscience Courses

Screenshot from Google, January 2023

I love everything about these sitelinks! The advertising is using them to reach people in all phases of the buyer journey.

For people not ready to commit:

• Study Neuroscience: This sitelink is broad and informational. It’s helpful to people who have just started to explore their options for studying neuroscience.
• Get Course Brochure: This sitelink is also great for people in the research phase. And while we mostly live in an online world, some people still prefer to consume hard-copy books, brochures, etc. With this sitelink, the school is covering its bases.

For people getting close to committing:

• Online Short Course: This is the course the school offers. It’s a great sitelink for those almost ready to sign up.

For people ready to sign up:

• Register Online Now: This is the strongest call to action for those ready to commit. It takes people directly to the signup page.

Example 3: Neuroscience Degrees

Let’s look at another example from the world of neuroscience education: this time for a neuroscience degree program.

Screenshot from Google, January 2023

In contrast to the previous two examples, the sitelinks in this ad aren’t as strong.

Academics Overview: This sitelink seems more appropriate for a broad term search, such as a search on the school’s name. If the searcher is looking for a specific degree program (which seems like the intention based on the term and the ad), the sitelinks should be something specific to that particular degree program.

Scholarships: Just as with the above sitelink, “Scholarships” doesn’t seem very helpful either. The topic of scholarships is important—but probably doesn’t need to be addressed until the person determines that this school is a good fit.

Example 4: Code Security

Next, let’s look at two Google search ads for code security products.

Screenshot from Google, January 2023

The sitelinks in these two ads look like typical assets you’d find for SaaS, cloud-based, or tech companies. They click through to a lot of helpful information, such as product plans and success stories.

I particularly like the Most Common Risks sitelink in the second ad. It leads to a helpful article that would be great for engaging top-of-funnel leads.

On the flip side, I’m not a big fan of the Blog sitelink in the first ad. “Blog” simply isn’t very descriptive or helpful.

Still, there are no right or wrong sitelinks here. And it would be interesting to test my theory that blog content is not a top-performing asset!

Sitelink Assets Are More Than An Afterthought

I hope I’ve convinced you of the usefulness and versatility of sitelinks when created with specific objectives that align with your broader strategy.

So don’t create your sitelink assets as an afterthought.

Because if you give them the careful consideration they deserve, they’ll serve you well.

Note: Google sitelink assets were previously known as sitelink extensions and renamed in September 2022.

More resources:

Featured Image: Thaspol Sangsee/Shutterstock