SEO
How to Use Keywords for SEO: The Complete Beginner’s Guide
In this guide, I’ll cover in detail how to make the best use of keywords in three steps:
- Finding good keywords: keywords that are rankable and bring value to your site.
- Using keywords in content and meta tags: how to use the target keyword to structure and write content that will satisfy readers and send relevance signals to search engines.
- Tracking keywords: monitoring your (and your competitors’) performance.
There’s really a lot you can do with just a single keyword, so at the end of the article, you’ll find a few advanced SEO tips.
Once you know how to find one good keyword, you will be able to create an entire list of keywords.
1. Pick relevant seed keywords to generate keyword ideas
Seed keywords are words or phrases that you can use as the starting point in a keyword research process to unlock more keywords. For example, for our site, these could be general terms like “seo, organic traffic, digital marketing, keywords, backlinks”, etc.
There are many good sources of seed keywords, and it’s not a bad idea to try them all:
- Brainstorming. This involves gathering a team or working solo to think deeply about the terms your potential customers might use when searching for your products or services.
- Your competitors’ website navigation. The labels they use in their navigation menus, headers, and footers often highlight critical industry terms and popular products or services that you might also want to target.
- Your competitors’ keywords. Tools like Ahrefs can help you discover which keywords your competitors are targeting in their SEO and paid ad campaigns. I’ll cover that in a bit.
- Your website and promo materials. Review your website’s text, especially high-performing pages, as well as any promotional materials like brochures, ads, and press releases. These sources can reveal the terms that already resonate with your audience.
- Generative AI. AI tools can generate keyword ideas based on brief descriptions of your business, products, or industry (example below).
Here’s what you can ask any generative AI for, whether that’s Copilot, ChatGPT, Perplexity, etc.
Next, paste your seed keywords into a tool like Ahrefs Keywords Explorer to generate keyword ideas. If you’re using Ahrefs, you can go straight into Keywords Explorer, get AI suggestions there, and start researching right away.
Next, make sure you’ve set up the country in which you’ll want to rank and hit “Search”.
After hitting the “Search” button, go to the Matching terms report. You will see a big list of keywords.
The list you’ll get will be quite raw — not all keywords will be equally good and the list will likely be too big to manage. Next steps are all about refining the list because we’ll be looking for target keywords — the keywords that will become the topic of your content.
2. Refine the list and cluster
The next step is to refine your list using filters.
Some useful basic filters are:
- KD (Keyword Difficulty): how difficult it would be to rank on the first page of Google for a given keyword.
- Traffic potential: traffic you can get for ranking #1 for that keyword and other relevant keywords (based on the page that currently ranks #1).
- Lowest DR (Domain Rating): plug in the DR of your site to see keywords where another site with the same DR already ranks in the top 10. In other words, it helps to find “rankable” keywords.
- Target: one of the main use cases is excluding keywords you already rank for.
- Include/Exclude: see keywords that contain specific words to increase relevancy/hide keywords with irrelevant words.
For example, here’s how to find potentially rankable keywords with traffic potential above 300 monthly visits. Go to the Matching terms report in Keywords Explorer and set filters: keyword difficulty filter (KD) to your site’s Domain Rating, Traffic potential, and Volume filters to a minimum of 300.
Clustering is another step to refine your list. It shows you if there is another keyword you could target to get more traffic (aka parent topic). At the same time, it shows which keywords most likely belong on the same page.
For example, here are some clusters distilled from low-competition topics about marketing.
Pro tip
Take keyword trends into account when choosing keywords.
For example, the keyword “is affiliate marketing legit” is at 8.8k monthly search volume right now, but based on our forecast in Keywords Explorer (the orange part of the chart), if it continues its current growth rate it should be more than triple next year.
The graphs will also show you if the search volume is affected by seasonality (fluctuations in search volume throughout different times of the year).
4. Identify search intent and determine value for your site
Before investing time in content, make sure you can give searchers what they want and that the keywords will attract the right kind of audience.
To identify the type of page you need to create to satisfy searchers, look at the top-ranking pages to see what purpose they serve (are they more informational or commercial), or simply use the Identify intents AI feature in Keywords Explorer.
So, for example, if the top-ranking pages are ecommerce pages and you’re not offering products on your site, it’s going to be very hard to rank.
Then, ask yourself if visitors attracted by a keyword will be valuable to your business — whether they’re likely to subscribe to your newsletter or make a purchase. At Ahrefs, we use a business potential score to evaluate this.
Finally, if a keyword checks all boxes, add it to a keyword list.
Now you’ve got a list of relevant, valuable target keywords with traffic potential ready for content creation. You can repeat the process as many times as you like with different seed keywords or different filters and find new ideas.
There’s one more great source of keywords — competitors.
5. Enrich the list with your competitors’ keywords
In this step, we’ll do a content gap analysis to find keywords your competitors already rank for, but you don’t.
First, let’s find your competitors.
- Enter your domain in Ahrefs’ Site Explorer and go to the Organic competitors report.
- Select the most relevant competitors and click on Copy (this copies URLs — we’ll use them in another tool).
Next, we’ll see which keywords you’re missing.
- Go to Ahrefs’ Competitive Analysis tool, paste the previously copied URLs, enter your domain on top and hit Show keyword opportunities.
- In the Content gap report, use filters to refine the report.
- Select keywords and add them to your list.
Pro tip
If you stumble across two similar keywords there’s an easy way to determine if they belong on the same page.
- Enter the keyword in Keywords Explorer.
- Scroll to SERP overview, click Compare with, and enter the keyword to compare with.
- Fewer common results and low SERP (Search Engine Result Page) similarity mean separate pages should target the two keywords.
Once you have your target keyword, you can include it in relevant places in your on-page content, including:
- Key elements of search intent (content type, format, and angle).
- URL slug.
- Title and H1.
- Meta description.
- Subheadings (H2 – H6).
- Main content.
- Anchor text for links.
And, just so we’re on the same page, the target keyword is the topic of the content and the main keyword you’ll be optimizing for and tracking later on.
Use the target keyword to determine the search intent
Search intent is the reason behind the search. Understanding it tells you what users are looking for and what you need to deliver in your content.
To identify search intent, look at the top-ranking results for your target keyword on Google and identify the three Cs of search intent:
- Content type – What is the dominating type of content? Is it a blog post, product page, video, or something else? If you’ve done that during the keyword research phase (highly recommended), only two elements to go.
- Content format – Some common formats include how-to guides, list posts, reviews, comparisons, etc.
- Content angle – The unique selling point of the top-ranking points, e.g., “best,” “cheapest,” “for beginners,” etc. Provides insight into what searchers value in a particular search.
For example, most top-ranking pages for “avocado seed” are blog posts serving as how-to guides for planting the seed. The use of easy and simple angles indicates that searchers are beginners looking for straightforward advice.
Use the target keyword in the URL slug
A URL slug is the part of the URL that identifies a specific page on a website in a form readable by both users and search engines.
If you look at the URL of the page you’re on, that will be the last part, “how-to-use-keywords-seo”.
https://ahrefs.com/blog/how-to-use-keywords-seo
Google says to use words that are relevant to your content inside page URLs (source). Usually, the easiest way to do that is to set your target keyword in the slug part of the URL.
Use the target keyword in the title and match it with the H1 tag
A title tag is a bit of HTML code used to specify the title of a webpage.
<title>How to Use Keywords for SEO: A-Z Guide For Beginners</title>
The H1 tag is an HTML heading that’s most commonly used to mark up a web page title.
<h1>How to Use Keywords for SEO: A-Z Guide For Beginners</h1>
Both are very important to Google and searchers. Since they both indicate what the page is about, you can just match them, like I did in this article.
Titles help Google understand the context of a page. What’s more, even a slight improvement to your title can improve your rankings.
Google advises focusing on creating good titles, which should be “unique to the page, clear and concise, and accurately describe the contents of the page” (source). It’s hard to think of a better way to accurately describe the contents other than using the target keyword.
If it makes sense for the title, aim for an exact match of the keyword. But if you need to insert a preposition or break the phrase, this won’t make Google think your page is less relevant. Google understands close variations of the keyword really well, so there’s no need to stuff in similar keywords, misspellings, etc.
To illustrate, my old article on how to see keywords that Google Analytics won’t show ranks #1 for many variations of the phrase in the title.
Use the keyword in meta description
However, don’t write meta descriptions solely for Google; Google rewrites them more than half of the time (study) and doesn’t use them for ranking purposes. Instead, focus on crafting meta descriptions for searchers.
These descriptions appear in the SERPs, where users can read them. If your description is relevant and compelling, it can increase the likelihood of users clicking on your link.
Including your target keyword in the meta description is usually natural. For instance, consider the description of the article mentioned earlier. Incorporating the keyword into the sentences simply provides a comprehensive way to describe the issue.
Use the target keyword to find secondary keywords
Secondary keywords are any keywords closely related to the primary keyword that you’re targeting with your page.
You can find them through your primary keyword and use them as subheadings (H2 to H6 tags) and talking points throughout the content. Here’s how.
Go to Keywords Explorer and plug in your target keyword. From there, head on to the Related terms report and toggle between:
- Also rank for: keywords that the top 10 ranking pages also rank for.
- Also talk about: keywords frequently mention by top-ranking articles.
Now, to know how to use these keywords in your text, just manually look at the top ranking pages and see how and where they cover the keywords.
For example, looking at one of the top articles for “digital marketing”, we can see right away that some of the most important aspects are the definition, a template and importance. You can use the free Ahrefs SEO Toolbar to break down the structure of any page instantly.
Another place you can look for inspiration is the People Also Ask Box in the SERPs. Use it to find words and subtopics that may be worth adding to the article.
Pro tip
Optimizing an existing article?
Use the Content Gap tool to find subtopics you may be missing. The tool shows keywords that your competitors’ pages rank for, and your page doesn’t.
- Go to Keywords Explorer and enter your target keyword.
- Scroll down to the SERP overview, select a few top pages, and click Open in Content Gap.
- In Content Gap, click on Targets and add the page you’re optimizing in the last field.
Use primary and secondary keywords in the main content
To rank high on search engines, it’s important to include your keywords in your text. Even though Google is good at understanding similar words and variations, it still helps to use the specific keywords people might search for. Google explains that in their short guide to how search works:
The most basic signal that information is relevant is when content contains the same keywords as your search query. For example, with webpages, if those keywords appear on the page, or if they appear in the headings or body of the text, the information might be more relevant.
When writing, it’s important to incorporate keywords naturally. Start your content with the most relevant information that people are likely to search for. This ensures that key points are immediately visible to your readers and search engines.
If you have a secondary keyword that’s less critical but still relevant, consider giving it a dedicated section. This approach allows you to explore the topic in detail, rather than briefly mentioning it at the end of your content.
However, avoid overemphasizing the frequency of your keywords. Effective SEO involves more than just repeating keywords. If SEO were simply about keyword density, it would be straightforward, but such strategies don’t lead to long-term success and can make your content feel spammy.
For instance, if ‘content strategy’ is a central theme of your discussion and you mention it only once, Google might perceive your content as incomplete. On the other hand, stuffing your article with the term ‘content strategy’ more than necessary won’t outperform your competitors and could potentially lead to your site being flagged as spam.
Use the target keyword in link anchor text and/or surrounding text
The anchor text or link text is the clickable text of an HTML hyperlink.
Google uses anchors to understand the page’s context. There even seems to be a consensus that anchor text is a ranking factor, although, according to our study, it is likely a weak one.
In situations like these, it’s just best to stick with Google’s advice:
Good anchor text is descriptive, reasonably concise, and relevant to the page that it’s on and to the page it links to. It provides context for the link, and sets the expectation for your readers. (…)
Remember to give context to your links: the words before and after links matter, so pay attention to the sentence as a whole.
So use the target keyword in the anchor text and or surrounding text but keep it natural — add only on pages that are related to the page you’re linking to and use text that will help the readers understand where and why you’re linking.
Rank tracking refers to monitoring the positions of a website’s pages in search engine results for specific keywords.
It’s pretty much an automated process; everything can be handled by a tool like Ahrefs’ Rank Tracker. No need to check rankings manually and note them down in a spreadsheet.
If you have a keyword list ready, all you need to do is add that list to Rank Tracker.
The keywords will appear in Rank Tracker’s Overview report.
Another way to add keywords is to hit Add keywords in the top right corner (best for adding single keywords or importing a list from a document).
Now to compare your performance against competitors, just go to the Competitors report. The metric I recommend tracking is SOV (Share of Voice). It shows how many clicks go to your pages compared to competitors.
One of the key advantages of SOV is that it accounts for fluctuations in search volume trends. Therefore, if you notice a decrease in traffic but maintain a high SOV, it indicates that the drop is due to a decrease in the overall popularity of the keywords, not a decline in your SEO effectiveness.
But not only can you track your competitors’ keywords, you can also monitor them. Use a tool like Ahrefs Alerts to get notifications whenever your competitors started working for a new keyword.
Just to go Alerts tool in Ahrefs and fill in the details.
There’s even more you can do with keywords and a bit more you should know to avoid some common mistakes.
1. Use keywords to find guest blogging opportunities
Guest blogging is the practice of writing and publishing a blog post on another person or company’s website.
It’s one of the most popular link building tactics with a few other benefits like exposure to a new, targeted audience.
Here’s how to find relevant, high-quality sites to pitch:
- Go to Ahrefs’ Content Explorer.
- Enter a broad keyword or phrase related to your niche.
- Select In title from the drop-down menu.
- Run the search.
Next, refine the list by applying these filters:
- Domain Rating (DR) from 30 to 60.
- Click the One page per domain filter.
- Click the Exclude homepages filter.
- Click the Exclude subdomains filter.
Finally, click on the Websites tab to see potential websites you could guest blog for.
2. Use keywords to find internal link opportunities
Internal links take visitors from one page to another on your website. Their main purpose is to help visitors easily navigate your website, but they can also help boost SEO by aiding the flow of link equity.
Finding new internal link opportunities is also a time-consuming process if done manually, but you can identify them in bulk using Ahrefs’ Site Audit. The tool takes the top 10 keywords (by traffic) for each crawled page, then looks for mentions of those on your other crawled pages.
Click on the Internal link opportunities report in Site Audit.
You’ll see a bunch of suggestions on how to improve your internal linking using new links. The tool even suggests exactly where to place the internal link.
3. Use keywords to find link building opportunities
Link building is the process of getting other websites to link to pages on your website. Its purpose is to boost the “authority” of your pages in the eyes of Google and help your pages rank.
A good place to start is to pull up the top-ranking pages for your target keyword and research where they got their links from.
Put your keyword into Keywords Explorer and scroll down to the SERP overview. You’ll see the top-ranking pages and their number of backlinks (and linking domains).
Once you click on any of the backlink numbers, you’ll be redirected to a list of backlinks of a given page in Site Explorer.
From that point, the typical process involves identifying sites with the highest potential to boost your SEO and contacting their owners if you think they’d be willing to link to your content. We’re covering the details of this process and everything else you need to know to start with link building in this guide.
4. Avoid common keyword pitfalls
Four big don’ts of using keywords.
- Don’t use the same keyword excessively on a page. Repeating a keyword too frequently within a single page can lead to keyword stuffing, which is treated as spam by Google.
- Don’t use the same focus keyword across multiple pages. Each page on your website should have a unique focus keyword. Using the same keyword across multiple pages can lead to keyword cannibalization, where pages compete against each other in search results.
- Don’t sacrifice quality content for keyword usage. While keywords are essential for SEO, prioritize high-quality, informative content above all else. Don’t make your content read unnatural or too long by cramming in keywords. This won’t help you rank and will decrease content quality.
- Don’t use keywords just for the sake of using them. This means two things. First, don’t target keywords not related to your website or business — this will only bring you useless traffic. Second, don’t try to hit some keyword frequency goal which is often suggested by content optimization tools by just mentioning the keyword without any substance — SEO doesn’t work that way anymore.
Final thoughts
This article focused on general SEO for text-based content. For using keywords in other types of content and SEO, see these guides:
SEO
How to Revive an Old Blog Article for SEO
Quick question: What do you typically do with your old blog posts? Most likely, the answer is: Not much.
If that’s the case, you’re not alone. Many of us in SEO and content marketing tend to focus on continuously creating new content, rather than leveraging our existing blog posts.
However, here’s the reality—Google is becoming increasingly sophisticated in evaluating content quality, and we need to adapt accordingly. Just as it’s easier to encourage existing customers to make repeat purchases, updating old content on your website is a more efficient and sustainable strategy in the long run.
Ways to Optimize Older Content
Some of your old content might not be optimized for SEO very well, rank for irrelevant keywords, or drive no traffic at all. If the quality is still decent, however, you should be able to optimize it properly with little effort.
Refresh Content
If your blog post contains a specific year or mentions current events, it may become outdated over time. If the rest of the content is still relevant (like if it’s targeting an evergreen topic), simply updating the date might be all you need to do.
Rewrite Old Blog Posts
When the content quality is low (you might have greatly improved your writing skills since you’ve written the post) but the potential is still there, there’s not much you can do apart from rewriting an old blog post completely.
This is not a waste—you’re saving time on brainstorming since the basic structure is already in place. Now, focus on improving the quality.
Delete Old Blog Posts
You might find a blog post that just seems unusable. Should you delete your old content? It depends. If it’s completely outdated, of low quality, and irrelevant to any valuable keywords for your website, it’s better to remove it.
Once you decide to delete the post, don’t forget to set up a 301 redirect to a related post or page, or to your homepage.
Promote Old Blog Posts
Sometimes all your content needs is a bit of promotion to start ranking and getting traffic again. Share it on your social media, link to it from a new post – do something to get it discoverable again to your audience. This can give it the boost it needs to attract organic links too.
Which Blog Posts Should You Update?
Deciding when to update or rewrite blog posts is a decision that relies on one important thing: a content audit.
Use your Google Analytics to find out which blog posts used to drive tons of traffic, but no longer have the same reach. You can also use Google Search Console to find out which of your blog posts have lost visibility in comparison to previous months. I have a guide on website analysis using Google Analytics and Google Search Console you can follow.
If you use keyword tracking tools like SE Ranking, you can also use the data it provides to come up with a list of blog posts that have dropped in the rankings.
Make data-driven decisions to identify which blog posts would benefit from these updates – i.e., which ones still have the chance to recover their keyword rankings and organic traffic.
With Google’s helpful content update, which emphasizes better user experiences, it’s crucial to ensure your content remains relevant, valuable, and up-to-date.
How To Update Old Blog Posts for SEO
Updating articles can be an involved process. Here are some tips and tactics to help you get it right.
Author’s Note: I have a Comprehensive On-Page SEO Checklist you might also be interested in following while you’re doing your content audit.
Conduct New Keyword Research
Updating your post without any guide won’t get you far. Always do your keyword research to understand how users are searching for your given topic.
Proper research can also show you relevant questions and sections that can be added to the blog post you’re updating or rewriting. Make sure to take a look at the People Also Ask (PAA) section that shows up when you search for your target keyword. Check out other websites like Answer The Public, Reddit, and Quora to see what users are looking for too.
Look for New Ranking Opportunities
When trying to revive an old blog post for SEO, keep an eye out for new SEO opportunities (e.g., AI Overview, featured snippets, and related search terms) that didn’t exist when you first wrote your blog post. Some of these features can be targeted by the new content you will add to your post, if you write with the aim to be eligible for it.
Rewrite Headlines and Meta Tags
If you want to attract new readers, consider updating your headlines and meta tags.
Your headlines and meta tags should fulfill these three things:
- Reflect the rewritten and new content you’ve added to the blog post.
- Be optimized for the new keywords it’s targeting (if any).
- Appeal to your target audience – who may have changed tastes from when the blog post was originally made.
Remember that your meta tags in particular act like a brief advertisement for your blog post, since this is what the user first sees when your blog post is shown in the search results page.
Take a look at your blog post’s click-through rate on Google Search Console – if it falls below 2%, it’s definitely time for new meta tags.
Replace Outdated Information and Statistics
Updating blog content with current studies and statistics enhances the relevance and credibility of your post. By providing up-to-date information, you help your audience make better, well-informed decisions, while also showing that your content is trustworthy.
Tighten or Expand Ideas
Your old content might be too short to provide real value to users – or you might have rambled on and on in your post. It’s important to evaluate whether you need to make your content more concise, or if you need to elaborate more.
Keep the following tips in mind as you refine your blog post’s ideas:
- Evaluate Helpfulness: Measure how well your content addresses your readers’ pain points. Aim to follow the E-E-A-T model (Experience, Expertise, Authoritativeness, Trustworthiness).
- Identify Missing Context: Consider whether your content needs more detail or clarification. View it from your audience’s perspective and ask if the information is complete, or if more information is needed.
- Interview Experts: Speak with industry experts or thought leaders to get fresh insights. This will help support your writing, and provide unique points that enhance the value of your content.
- Use Better Examples: Examples help simplify complex concepts. Add new examples or improve existing ones to strengthen your points.
- Add New Sections if Needed: If your content lacks depth or misses a key point, add new sections to cover these areas more thoroughly.
- Remove Fluff: Every sentence should contribute to the overall narrative. Eliminate unnecessary content to make your post more concise.
- Revise Listicles: Update listicle items based on SEO recommendations and content quality. Add or remove headings to stay competitive with higher-ranking posts.
Improve Visuals and Other Media
No doubt that there are tons of old graphics and photos in your blog posts that can be improved with the tools we have today. Make sure all of the visuals used in your content are appealing and high quality.
Update Internal and External Links
Are your internal and external links up to date? They need to be for your SEO and user experience. Outdated links can lead to broken pages or irrelevant content, frustrating readers and hurting your site’s performance.
You need to check for any broken links on your old blog posts, and update them ASAP. Updating your old blog posts can also lead to new opportunities to link internally to other blog posts and pages, which may not have been available when the post was originally published.
Optimize for Conversions
When updating content, the ultimate goal is often to increase conversions. However, your conversion goals may have changed over the years.
So here’s what you need to check in your updated blog post. First, does the call-to-action (CTA) still link to the products or services you want to promote? If not, update it to direct readers to the current solution or offer.
Second, consider where you can use different conversion strategies. Don’t just add a CTA at the end of the post.
Last, make sure that the blog post leverages product-led content. It’s going to help you mention your products and services in a way that feels natural, without being too pushy. Being subtle can be a high ROI tactic for updated posts.
Key Takeaway
Reviving old blog articles for SEO is a powerful strategy that can breathe new life into your content and boost your website’s visibility. Instead of solely focusing on creating new posts, taking the time to refresh existing content can yield impressive results, both in terms of traffic and conversions.
By implementing these strategies, you can transform old blog posts into valuable resources that attract new readers and retain existing ones. So, roll up your sleeves, dive into your archives, and start updating your content today—your audience and search rankings will thank you!
SEO
How Compression Can Be Used To Detect Low Quality Pages
The concept of Compressibility as a quality signal is not widely known, but SEOs should be aware of it. Search engines can use web page compressibility to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords, making it useful knowledge for SEO.
Although the following research paper demonstrates a successful use of on-page features for detecting spam, the deliberate lack of transparency by search engines makes it difficult to say with certainty if search engines are applying this or similar techniques.
What Is Compressibility?
In computing, compressibility refers to how much a file (data) can be reduced in size while retaining essential information, typically to maximize storage space or to allow more data to be transmitted over the Internet.
TL/DR Of Compression
Compression replaces repeated words and phrases with shorter references, reducing the file size by significant margins. Search engines typically compress indexed web pages to maximize storage space, reduce bandwidth, and improve retrieval speed, among other reasons.
This is a simplified explanation of how compression works:
- Identify Patterns:
A compression algorithm scans the text to find repeated words, patterns and phrases - Shorter Codes Take Up Less Space:
The codes and symbols use less storage space then the original words and phrases, which results in a smaller file size. - Shorter References Use Less Bits:
The “code” that essentially symbolizes the replaced words and phrases uses less data than the originals.
A bonus effect of using compression is that it can also be used to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords.
Research Paper About Detecting Spam
This research paper is significant because it was authored by distinguished computer scientists known for breakthroughs in AI, distributed computing, information retrieval, and other fields.
Marc Najork
One of the co-authors of the research paper is Marc Najork, a prominent research scientist who currently holds the title of Distinguished Research Scientist at Google DeepMind. He’s a co-author of the papers for TW-BERT, has contributed research for increasing the accuracy of using implicit user feedback like clicks, and worked on creating improved AI-based information retrieval (DSI++: Updating Transformer Memory with New Documents), among many other major breakthroughs in information retrieval.
Dennis Fetterly
Another of the co-authors is Dennis Fetterly, currently a software engineer at Google. He is listed as a co-inventor in a patent for a ranking algorithm that uses links, and is known for his research in distributed computing and information retrieval.
Those are just two of the distinguished researchers listed as co-authors of the 2006 Microsoft research paper about identifying spam through on-page content features. Among the several on-page content features the research paper analyzes is compressibility, which they discovered can be used as a classifier for indicating that a web page is spammy.
Detecting Spam Web Pages Through Content Analysis
Although the research paper was authored in 2006, its findings remain relevant to today.
Then, as now, people attempted to rank hundreds or thousands of location-based web pages that were essentially duplicate content aside from city, region, or state names. Then, as now, SEOs often created web pages for search engines by excessively repeating keywords within titles, meta descriptions, headings, internal anchor text, and within the content to improve rankings.
Section 4.6 of the research paper explains:
“Some search engines give higher weight to pages containing the query keywords several times. For example, for a given query term, a page that contains it ten times may be higher ranked than a page that contains it only once. To take advantage of such engines, some spam pages replicate their content several times in an attempt to rank higher.”
The research paper explains that search engines compress web pages and use the compressed version to reference the original web page. They note that excessive amounts of redundant words results in a higher level of compressibility. So they set about testing if there’s a correlation between a high level of compressibility and spam.
They write:
“Our approach in this section to locating redundant content within a page is to compress the page; to save space and disk time, search engines often compress web pages after indexing them, but before adding them to a page cache.
…We measure the redundancy of web pages by the compression ratio, the size of the uncompressed page divided by the size of the compressed page. We used GZIP …to compress pages, a fast and effective compression algorithm.”
High Compressibility Correlates To Spam
The results of the research showed that web pages with at least a compression ratio of 4.0 tended to be low quality web pages, spam. However, the highest rates of compressibility became less consistent because there were fewer data points, making it harder to interpret.
Figure 9: Prevalence of spam relative to compressibility of page.
The researchers concluded:
“70% of all sampled pages with a compression ratio of at least 4.0 were judged to be spam.”
But they also discovered that using the compression ratio by itself still resulted in false positives, where non-spam pages were incorrectly identified as spam:
“The compression ratio heuristic described in Section 4.6 fared best, correctly identifying 660 (27.9%) of the spam pages in our collection, while misidentifying 2, 068 (12.0%) of all judged pages.
Using all of the aforementioned features, the classification accuracy after the ten-fold cross validation process is encouraging:
95.4% of our judged pages were classified correctly, while 4.6% were classified incorrectly.
More specifically, for the spam class 1, 940 out of the 2, 364 pages, were classified correctly. For the non-spam class, 14, 440 out of the 14,804 pages were classified correctly. Consequently, 788 pages were classified incorrectly.”
The next section describes an interesting discovery about how to increase the accuracy of using on-page signals for identifying spam.
Insight Into Quality Rankings
The research paper examined multiple on-page signals, including compressibility. They discovered that each individual signal (classifier) was able to find some spam but that relying on any one signal on its own resulted in flagging non-spam pages for spam, which are commonly referred to as false positive.
The researchers made an important discovery that everyone interested in SEO should know, which is that using multiple classifiers increased the accuracy of detecting spam and decreased the likelihood of false positives. Just as important, the compressibility signal only identifies one kind of spam but not the full range of spam.
The takeaway is that compressibility is a good way to identify one kind of spam but there are other kinds of spam that aren’t caught with this one signal. Other kinds of spam were not caught with the compressibility signal.
This is the part that every SEO and publisher should be aware of:
“In the previous section, we presented a number of heuristics for assaying spam web pages. That is, we measured several characteristics of web pages, and found ranges of those characteristics which correlated with a page being spam. Nevertheless, when used individually, no technique uncovers most of the spam in our data set without flagging many non-spam pages as spam.
For example, considering the compression ratio heuristic described in Section 4.6, one of our most promising methods, the average probability of spam for ratios of 4.2 and higher is 72%. But only about 1.5% of all pages fall in this range. This number is far below the 13.8% of spam pages that we identified in our data set.”
So, even though compressibility was one of the better signals for identifying spam, it still was unable to uncover the full range of spam within the dataset the researchers used to test the signals.
Combining Multiple Signals
The above results indicated that individual signals of low quality are less accurate. So they tested using multiple signals. What they discovered was that combining multiple on-page signals for detecting spam resulted in a better accuracy rate with less pages misclassified as spam.
The researchers explained that they tested the use of multiple signals:
“One way of combining our heuristic methods is to view the spam detection problem as a classification problem. In this case, we want to create a classification model (or classifier) which, given a web page, will use the page’s features jointly in order to (correctly, we hope) classify it in one of two classes: spam and non-spam.”
These are their conclusions about using multiple signals:
“We have studied various aspects of content-based spam on the web using a real-world data set from the MSNSearch crawler. We have presented a number of heuristic methods for detecting content based spam. Some of our spam detection methods are more effective than others, however when used in isolation our methods may not identify all of the spam pages. For this reason, we combined our spam-detection methods to create a highly accurate C4.5 classifier. Our classifier can correctly identify 86.2% of all spam pages, while flagging very few legitimate pages as spam.”
Key Insight:
Misidentifying “very few legitimate pages as spam” was a significant breakthrough. The important insight that everyone involved with SEO should take away from this is that one signal by itself can result in false positives. Using multiple signals increases the accuracy.
What this means is that SEO tests of isolated ranking or quality signals will not yield reliable results that can be trusted for making strategy or business decisions.
Takeaways
We don’t know for certain if compressibility is used at the search engines but it’s an easy to use signal that combined with others could be used to catch simple kinds of spam like thousands of city name doorway pages with similar content. Yet even if the search engines don’t use this signal, it does show how easy it is to catch that kind of search engine manipulation and that it’s something search engines are well able to handle today.
Here are the key points of this article to keep in mind:
- Doorway pages with duplicate content is easy to catch because they compress at a higher ratio than normal web pages.
- Groups of web pages with a compression ratio above 4.0 were predominantly spam.
- Negative quality signals used by themselves to catch spam can lead to false positives.
- In this particular test, they discovered that on-page negative quality signals only catch specific types of spam.
- When used alone, the compressibility signal only catches redundancy-type spam, fails to detect other forms of spam, and leads to false positives.
- Combing quality signals improves spam detection accuracy and reduces false positives.
- Search engines today have a higher accuracy of spam detection with the use of AI like Spam Brain.
Read the research paper, which is linked from the Google Scholar page of Marc Najork:
Detecting spam web pages through content analysis
Featured Image by Shutterstock/pathdoc
SEO
New Google Trends SEO Documentation
Google Search Central published new documentation on Google Trends, explaining how to use it for search marketing. This guide serves as an easy to understand introduction for newcomers and a helpful refresher for experienced search marketers and publishers.
The new guide has six sections:
- About Google Trends
- Tutorial on monitoring trends
- How to do keyword research with the tool
- How to prioritize content with Trends data
- How to use Google Trends for competitor research
- How to use Google Trends for analyzing brand awareness and sentiment
The section about monitoring trends advises there are two kinds of rising trends, general and specific trends, which can be useful for developing content to publish on a site.
Using the Explore tool, you can leave the search box empty and view the current rising trends worldwide or use a drop down menu to focus on trends in a specific country. Users can further filter rising trends by time periods, categories and the type of search. The results show rising trends by topic and by keywords.
To search for specific trends users just need to enter the specific queries and then filter them by country, time, categories and type of search.
The section called Content Calendar describes how to use Google Trends to understand which content topics to prioritize.
Google explains:
“Google Trends can be helpful not only to get ideas on what to write, but also to prioritize when to publish it. To help you better prioritize which topics to focus on, try to find seasonal trends in the data. With that information, you can plan ahead to have high quality content available on your site a little before people are searching for it, so that when they do, your content is ready for them.”
Read the new Google Trends documentation:
Get started with Google Trends
Featured Image by Shutterstock/Luis Molinero