SEO
Top 7 SEO Keyword Research Tools For Agencies
All successful SEO campaigns rely on accurate, comprehensive data. And that process starts with the right keyword research tools.
Sure, you can get away with collecting keyword data manually on your own. But while you may be saving the cost of a premium tool, manual keyword research costs you in ot
her ways:
- Efficiency. Doing keyword research manually is time intensive. How much is an hour of your time worth?
- Comprehensiveness. Historical and comprehensive data isn’t easy to get on your own. It’s too easy to miss out on vital information that will make your SEO strategy a success.
- Competition. Keyword research tools allow you to understand not only what users are searching for but also what your competition focuses on. You can quickly identify gaps and find the best path to profitability and success.
- Knowledge. Long-time SEO experts can craft their own keyword strategies with a careful analysis of the SERPs, but that requires years of practice, trial, and costly errors. Not everyone has that experience. And not everyone has made enough mistakes to avoid the pitfalls.
A good SEO keyword research tool eliminates much of the guesswork. Here are seven well-known and time-tested tools for SEO that will get you well on the way to dominating your market.
1. Google Keyword Planner
Cost: Free.
Google Keyword Planner is a classic favorite.
It’s free, but because the information comes directly from the search engine, it’s reliable and trustworthy. It’s also flexible, allowing you to:
- Identify new keywords.
- Find related keywords.
- Estimate the number of searches for each variation.
- Estimate competition levels.
The tool is easy to access and available as a web application and via API, and it costs nothing; it just requires a Google Ads account.
You must also be aware of a few things when using this tool.
First, these are estimates based on historical data. That means if trends change, it won’t necessarily be reflected here.
Google Keyword Planner also can’t tell you much about the SERP itself, such as what features you can capitalize on and how the feature converts.
Because it’s part of Google Ads, PPC experience can help you gain more insights. You’ll find trends broadly across a demographic or granular level, like a city, region, or major city.
Google Keyword Planner also tends to combine data for similar keywords. So, if you want to know if [keyword near me] is better than [keywords near me], you’ll need a different tool.
Lastly, the tool uses broad definitions of words like “competition,” which doesn’t tell you who is ranking for the term, how much they’re investing to hold that ranking, or how likely you are to unseat them from their coveted top 10 rankings.
That being said, it’s an excellent tool if you just want to get a quick look or fresh ideas, if you’d like to use an API and create your own tools, or simply prefer to do the other tasks yourself.
2. Keyword.io
Cost: Free, $29 per month, and $49 per month.
If Google’s Keyword Planner isn’t quite enough, but you’re on a tight budget, Keyword.io may be the alternative you need. It also has different features.
Keyword.io uses autocomplete APIs to pull basic data for several sites and search engines, including Google, Amazon, eBay, Bing, Wikipedia, Alibaba, YouTube, Yandex, Fiverr, and Fotolia. This is perfect for niche clients and meeting specific needs.
It also has a Question/Intent Generator, an interactive topic explorer, and a topical overview tool.
In its user interface (UI), you’ll find an easy-to-use filter system and a chart that includes the competition, search volume, CPC, and a few other details about your chosen keywords.
It does have some limits, however.
You can run up to 20,000 keywords per seed with a limit of 100 requests per day (five per minute) or 1,000 requests per day (10 per minute) on its paid plans.
Its API access, related keywords tool, Google Ad data, and other features are also limited to paid accounts.
3. Semrush
Cost: $119.95 to $449.95 per month.
In its digital marketing suite, Semrush offers a collection of six keyword tools and four competitive analysis tools with a database of more than 21 billion keywords.
You can get a full overview of the keywords you’re watching, including paid and organic search volume, intent, competition, CPC, historical data, SERP analysis, and more.
You’ll get related keywords and questions, as well as a ton of guidance, ideas, and suggestions from the Semrush Magic, Position Tracking, and Organic Traffic Insights tools.
The Keyword Planner, however, is where much of the magic happens.
The organic competitor tab makes it easy to spot content and keyword gaps. Expand them and develop clusters that will help you grab traffic and conversions.
You can also see long-tail keyword data and other data to see what Page 1 holds regarding competition, difficulty, and opportunities at a broad or hyperlocal level.
The full suite of tools is a huge benefit. Teams can collaborate, share insights, and plan.
The seamless integration allows you to integrate your data, meaning teams can easily collaborate, share insights, and strategize.
And when you’re done, it can track everything you need for a successful digital marketing strategy.
Some of the tools they offer include:
- On-page SEO tools.
- Competitive analysis suite.
- Log file analysis.
- Site auditing.
- Content marketing tools.
- Marketing analysis.
- Paid advertising tools.
- Local SEO tools.
- Rank tracking.
- Social media management.
- Link-building tools.
- Amazon marketing tools.
- Website monetization tools.
Semrush’s best features when it comes to keyword research are its historical information and PPC metrics.
You can deep dive into campaigns and keywords to unlock the secrets of the SERPs and provide agency or in-house teams with priceless information they don’t usually access.
4. Moz Keyword Explorer
Cost: Free for 10 queries per month. $99-$599 per month.
With a database of more than 500 million keywords, Moz Keyword Explorer may be a great option if you’re looking to build a strategy rather than get a quick view of the data for a few keywords.
Moz has long been a leader in the SEO space.
Constantly updating and improving its Keyword Explorer Tool and its other core services, Moz keeps up with the trends and is well known for providing SEO professionals with the latest tools. And it has done so for more than a decade.
Like the Google Keyword Tool, Moz’s keyword planning tool provides information on the difficulty and monthly search volume for terms. It also lets you drill down geographically.
When you start, you’ll find the Keyword Overview, which provides monthly search volumes, ranking difficulty, organic click-through opportunities, and an estimated priority level.
You can also:
- Find new relevant keywords you should be targeting but aren’t.
- Learn how your site performs for keywords.
- Find areas where you can improve your SEO (including quick wins and larger investments).
- Prioritize keywords for efficient strategy creation.
- Top SERP analysis and features.
- Competitor analysis.
- Organic click-through rates.
Unlike the Google Keyword Tool, however, Moz supplies you with data beyond the basics. Think of it like keyword research and SERP analysis.
Moz does tend to have fewer keyword suggestions. And like Google’s Keyword Planner, it provides range estimates for search data rather than a specific number.
However, the database is updated frequently, so you can feel confident that you’re keeping up with the constant change in consumer search habits and rankings.
Plus, it’s easy to use, so teams can quickly take care of marketing tasks like finding opportunities, tracking performance, identifying problem areas, and gathering page-level details.
Moz also offers several other tools to help you get your site on track and ahead of the competition, but we really like it for its keyword research and flexibility.
5. Ahrefs Keyword Explorer
Cost: $99-$999 per month.
If I had to describe Ahrefs in one word, it would be power.
Enter a word into the search box, and you’re presented with multiple panels that can tell you everything you want to know about that keyword.
Total search volume, clicks, difficulty, the SERP features, and even a volume-difficulty distribution. And while it may look like a lot, all the information is well-organized and clearly presented.
Ahrefs provides terms in a parent-child topic format, providing the terms with context, so you can easily learn more about the terms, such as intent, while identifying overlap and keeping it all easy to find and understand.
These topics appear when you search for a related term, including the term’s ranking on the SERP, SERP result type, first-page ranking difficulty scores, and a snapshot of the user-delivered SERP. You can stay broad or narrow it all down by city or language.
Ahrefs can get a bit expensive. Agencies may find it difficult to scale if they prefer several user or client accounts, but it’s still one of the best and most reliable keyword research tools on the market.
What I really like about Ahrefs is that it’s thorough. It has one of the largest databases of all the tools available (19.2 billion keywords, 10 search engines, and 242 countries at the time of writing), and it’s regularly updated.
It makes international SEO strategies a breeze and includes data for everything from Google and Bing to YouTube and Amazon.
Plus, they clearly explain their metrics and database. And that level of transparency means trust.
Other tools in the suite include:
- Site Explorer.
- Site auditing.
- Rank tracking.
- Content Explorer.
6. SERanking
Cost: $23.52-$239 per month, depending on the ranking check and payment frequency.
SERanking shines as a keyword research tool within an all-around SEO toolkit. SERanking helps you keep costs down while offering features that allow agencies to meet clients’ unique needs.
One of the first things you’ll notice when you log in is its intuitive user interface. But this tool isn’t just another pretty online tool.
Its database is robust.
SERanking’s U.S. database includes 887 million keywords, 327 million U.S. domains, and 3 trillion indexed backlinks. And this doesn’t include its expansive European and Asian databases.
The overview page provides a solid look at the data, which includes search volume, the CPC, and a difficulty score.
SERanking also provides lists of related and low-volume keywords if you need inspiration or suggestions, as well as long-tail keyword suggestions with information about SERP features, competition levels, search volume, and other details you need to know to identify new opportunities.
Of course, identifying keywords is only the start of the mystery. How do you turn keywords into conversions? SERanking provides keyword tools that help you answer this question.
You can find out who the competition is in the organic results and see who is buying search ads, as well as details like estimated traffic levels and copies of the ads they’re using.
This allows you to see what’s working, gain insights into the users searching for those terms, and generate new ideas to try.
SERanking offers agency features, such as white labeling, report builders, lead generator, and other features you’ll find helpful.
However, one of the features agencies might find most helpful in keyword research is SERanking’s bulk keyword analysis, which lets you run thousands of keywords and download full reports for all the terms that matter.
Other tools in the SERanking Suite include:
- Keyword Rank Tracker.
- Keyword Grouper.
- Keyword Suggestions and Search Volume Checker.
- Index Status checker.
- Backlink Checker.
- Backlink monitoring.
- Competitive research tool.
- Website auditing tool.
- On-page SEO Checker.
- Page Changes Monitor.
- Social media analytics.
- Traffic analysis.
SERanking is more affordable than some of the other tools out there, but it does come at a cost.
It isn’t as robust as some of its competitors and doesn’t get as granular in the same way, but it still provides the features and data you need to create a successful SEO strategy.
And with its flexible pricing, this tool is well worth considering.
7. BrightEdge Data Cube
Cost: Custom pricing model.
If you’re looking for an AI-powered digital marketing tool suite that includes a quality research tool, BrightEdge may be the right option for you.
Unlike other tools that focus on supplying you with data and ways to analyze that data, BrightEdge looks to do much of the time-consuming analysis for you.
Among its search, content, social, local, and mobile solutions, you’ll find Data Cube – an AI-backed content and keyword tool that uses natural language processing to find related topics and keywords.
You’ll also encounter DataMind, an AI that helps you find search trends, changes in consumer behaviors, and important competitor movements you need to know about.
The two together make it quick and easy to perform keyword research, build out topics, create content strategies, and strengthen your SEO plans.
Once you enter a topic or broad keyword, the tool will provide you with relevant keywords, the search volume, competition levels, keyword value, it’s universal listing, and the number of words in the phrase.
Filter the results by a custom set of criteria to narrow the list down and get the necessary information.
Once you have a list, select the ones you want to keep and download them or use them with BrightEdge’s other tools to create full strategies and gain more insights.
This could include competitor analysis, analyzing SERP features, intent, or other tasks.
For agencies that provide local SEO, BrightEdge also offers HyperLocal, which helps you find and track keywords and keyword performance at the local level.
When you’re done, give the Opportunity Forecasting and tracking tools a try to monitor your progress and provide clients with the information they care about.
Perhaps the nicest feature for agencies is its Storybuilder – a reporting tool that allows you to create rich client reports that provide clients with targeted overviews and the data they’re most interested in.
If this sounds like the right tool for you, the company gives demos, but there are a few things you should consider.
First, it only updates once per month. And while the company keeps its pricing close to the chest, this digital marketing tool suite is a significant investment. It may not be the best choice if keyword research is the only thing you need.
Secondly, while the tools are highly sophisticated and refined, there is a learning curve to get started.
You’ll also discover that there are limits on features like keyword tracking, and it can be time-consuming to set up, with some adjustments requiring technical support.
Lastly, BrightEdge’s keyword research tool doesn’t let you get too far into the weeds and doesn’t include PPC traffic.
That aside, agencies and larger brands will find that it scales easily, has a beautifully designed UI, and makes you look great to clients.
The Best Agency SEO Keyword Research Tools
This list only contains seven of the many tools available today to help you get your keyword research done to an expert degree.
But no matter how many of the tools we share with you or which ones, it’s important to understand that none are flawless.
Each tool has its own unique strengths and weaknesses, so selecting a platform is very much dependent on the types of clients that you typically work with and personal preference.
In reality, you’ll likely find that you prefer to work between a few tools to accomplish everything you’d like.
Google Keyword Planner and Keyword.io are top choices when you want a quick look at the data, or you’d like to export the data to work on elsewhere. You may even want to use this data with the other tools mentioned in this chapter.
Ahrefs, Moz, Semrush, and BrightEdge are far more robust and are better suited to agency SEO tasks.
While not free (although they offer free plans or a trial period except BrightEdge), they allow you to really dig into the search space, ultimately resulting in higher traffic, more conversions, and stronger SEO strategies. These benefits require more time and often come with a learning curve.
By far, the most important keyword research tool you have access to is you.
Keyword research is more than simply choosing the keywords with the biggest search volume or the phrase with the lowest Cost Per Click (CPC).
It’s your expertise, experience, knowledge, and insights that transform data into digital marketing you can be proud of.
Featured Image: Paulo Bobita/Search Engine Journal
SEO
How to Revive an Old Blog Article for SEO
Quick question: What do you typically do with your old blog posts? Most likely, the answer is: Not much.
If that’s the case, you’re not alone. Many of us in SEO and content marketing tend to focus on continuously creating new content, rather than leveraging our existing blog posts.
However, here’s the reality—Google is becoming increasingly sophisticated in evaluating content quality, and we need to adapt accordingly. Just as it’s easier to encourage existing customers to make repeat purchases, updating old content on your website is a more efficient and sustainable strategy in the long run.
Ways to Optimize Older Content
Some of your old content might not be optimized for SEO very well, rank for irrelevant keywords, or drive no traffic at all. If the quality is still decent, however, you should be able to optimize it properly with little effort.
Refresh Content
If your blog post contains a specific year or mentions current events, it may become outdated over time. If the rest of the content is still relevant (like if it’s targeting an evergreen topic), simply updating the date might be all you need to do.
Rewrite Old Blog Posts
When the content quality is low (you might have greatly improved your writing skills since you’ve written the post) but the potential is still there, there’s not much you can do apart from rewriting an old blog post completely.
This is not a waste—you’re saving time on brainstorming since the basic structure is already in place. Now, focus on improving the quality.
Delete Old Blog Posts
You might find a blog post that just seems unusable. Should you delete your old content? It depends. If it’s completely outdated, of low quality, and irrelevant to any valuable keywords for your website, it’s better to remove it.
Once you decide to delete the post, don’t forget to set up a 301 redirect to a related post or page, or to your homepage.
Promote Old Blog Posts
Sometimes all your content needs is a bit of promotion to start ranking and getting traffic again. Share it on your social media, link to it from a new post – do something to get it discoverable again to your audience. This can give it the boost it needs to attract organic links too.
Which Blog Posts Should You Update?
Deciding when to update or rewrite blog posts is a decision that relies on one important thing: a content audit.
Use your Google Analytics to find out which blog posts used to drive tons of traffic, but no longer have the same reach. You can also use Google Search Console to find out which of your blog posts have lost visibility in comparison to previous months. I have a guide on website analysis using Google Analytics and Google Search Console you can follow.
If you use keyword tracking tools like SE Ranking, you can also use the data it provides to come up with a list of blog posts that have dropped in the rankings.
Make data-driven decisions to identify which blog posts would benefit from these updates – i.e., which ones still have the chance to recover their keyword rankings and organic traffic.
With Google’s helpful content update, which emphasizes better user experiences, it’s crucial to ensure your content remains relevant, valuable, and up-to-date.
How To Update Old Blog Posts for SEO
Updating articles can be an involved process. Here are some tips and tactics to help you get it right.
Author’s Note: I have a Comprehensive On-Page SEO Checklist you might also be interested in following while you’re doing your content audit.
Conduct New Keyword Research
Updating your post without any guide won’t get you far. Always do your keyword research to understand how users are searching for your given topic.
Proper research can also show you relevant questions and sections that can be added to the blog post you’re updating or rewriting. Make sure to take a look at the People Also Ask (PAA) section that shows up when you search for your target keyword. Check out other websites like Answer The Public, Reddit, and Quora to see what users are looking for too.
Look for New Ranking Opportunities
When trying to revive an old blog post for SEO, keep an eye out for new SEO opportunities (e.g., AI Overview, featured snippets, and related search terms) that didn’t exist when you first wrote your blog post. Some of these features can be targeted by the new content you will add to your post, if you write with the aim to be eligible for it.
Rewrite Headlines and Meta Tags
If you want to attract new readers, consider updating your headlines and meta tags.
Your headlines and meta tags should fulfill these three things:
- Reflect the rewritten and new content you’ve added to the blog post.
- Be optimized for the new keywords it’s targeting (if any).
- Appeal to your target audience – who may have changed tastes from when the blog post was originally made.
Remember that your meta tags in particular act like a brief advertisement for your blog post, since this is what the user first sees when your blog post is shown in the search results page.
Take a look at your blog post’s click-through rate on Google Search Console – if it falls below 2%, it’s definitely time for new meta tags.
Replace Outdated Information and Statistics
Updating blog content with current studies and statistics enhances the relevance and credibility of your post. By providing up-to-date information, you help your audience make better, well-informed decisions, while also showing that your content is trustworthy.
Tighten or Expand Ideas
Your old content might be too short to provide real value to users – or you might have rambled on and on in your post. It’s important to evaluate whether you need to make your content more concise, or if you need to elaborate more.
Keep the following tips in mind as you refine your blog post’s ideas:
- Evaluate Helpfulness: Measure how well your content addresses your readers’ pain points. Aim to follow the E-E-A-T model (Experience, Expertise, Authoritativeness, Trustworthiness).
- Identify Missing Context: Consider whether your content needs more detail or clarification. View it from your audience’s perspective and ask if the information is complete, or if more information is needed.
- Interview Experts: Speak with industry experts or thought leaders to get fresh insights. This will help support your writing, and provide unique points that enhance the value of your content.
- Use Better Examples: Examples help simplify complex concepts. Add new examples or improve existing ones to strengthen your points.
- Add New Sections if Needed: If your content lacks depth or misses a key point, add new sections to cover these areas more thoroughly.
- Remove Fluff: Every sentence should contribute to the overall narrative. Eliminate unnecessary content to make your post more concise.
- Revise Listicles: Update listicle items based on SEO recommendations and content quality. Add or remove headings to stay competitive with higher-ranking posts.
Improve Visuals and Other Media
No doubt that there are tons of old graphics and photos in your blog posts that can be improved with the tools we have today. Make sure all of the visuals used in your content are appealing and high quality.
Update Internal and External Links
Are your internal and external links up to date? They need to be for your SEO and user experience. Outdated links can lead to broken pages or irrelevant content, frustrating readers and hurting your site’s performance.
You need to check for any broken links on your old blog posts, and update them ASAP. Updating your old blog posts can also lead to new opportunities to link internally to other blog posts and pages, which may not have been available when the post was originally published.
Optimize for Conversions
When updating content, the ultimate goal is often to increase conversions. However, your conversion goals may have changed over the years.
So here’s what you need to check in your updated blog post. First, does the call-to-action (CTA) still link to the products or services you want to promote? If not, update it to direct readers to the current solution or offer.
Second, consider where you can use different conversion strategies. Don’t just add a CTA at the end of the post.
Last, make sure that the blog post leverages product-led content. It’s going to help you mention your products and services in a way that feels natural, without being too pushy. Being subtle can be a high ROI tactic for updated posts.
Key Takeaway
Reviving old blog articles for SEO is a powerful strategy that can breathe new life into your content and boost your website’s visibility. Instead of solely focusing on creating new posts, taking the time to refresh existing content can yield impressive results, both in terms of traffic and conversions.
By implementing these strategies, you can transform old blog posts into valuable resources that attract new readers and retain existing ones. So, roll up your sleeves, dive into your archives, and start updating your content today—your audience and search rankings will thank you!
SEO
How Compression Can Be Used To Detect Low Quality Pages
The concept of Compressibility as a quality signal is not widely known, but SEOs should be aware of it. Search engines can use web page compressibility to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords, making it useful knowledge for SEO.
Although the following research paper demonstrates a successful use of on-page features for detecting spam, the deliberate lack of transparency by search engines makes it difficult to say with certainty if search engines are applying this or similar techniques.
What Is Compressibility?
In computing, compressibility refers to how much a file (data) can be reduced in size while retaining essential information, typically to maximize storage space or to allow more data to be transmitted over the Internet.
TL/DR Of Compression
Compression replaces repeated words and phrases with shorter references, reducing the file size by significant margins. Search engines typically compress indexed web pages to maximize storage space, reduce bandwidth, and improve retrieval speed, among other reasons.
This is a simplified explanation of how compression works:
- Identify Patterns:
A compression algorithm scans the text to find repeated words, patterns and phrases - Shorter Codes Take Up Less Space:
The codes and symbols use less storage space then the original words and phrases, which results in a smaller file size. - Shorter References Use Less Bits:
The “code” that essentially symbolizes the replaced words and phrases uses less data than the originals.
A bonus effect of using compression is that it can also be used to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords.
Research Paper About Detecting Spam
This research paper is significant because it was authored by distinguished computer scientists known for breakthroughs in AI, distributed computing, information retrieval, and other fields.
Marc Najork
One of the co-authors of the research paper is Marc Najork, a prominent research scientist who currently holds the title of Distinguished Research Scientist at Google DeepMind. He’s a co-author of the papers for TW-BERT, has contributed research for increasing the accuracy of using implicit user feedback like clicks, and worked on creating improved AI-based information retrieval (DSI++: Updating Transformer Memory with New Documents), among many other major breakthroughs in information retrieval.
Dennis Fetterly
Another of the co-authors is Dennis Fetterly, currently a software engineer at Google. He is listed as a co-inventor in a patent for a ranking algorithm that uses links, and is known for his research in distributed computing and information retrieval.
Those are just two of the distinguished researchers listed as co-authors of the 2006 Microsoft research paper about identifying spam through on-page content features. Among the several on-page content features the research paper analyzes is compressibility, which they discovered can be used as a classifier for indicating that a web page is spammy.
Detecting Spam Web Pages Through Content Analysis
Although the research paper was authored in 2006, its findings remain relevant to today.
Then, as now, people attempted to rank hundreds or thousands of location-based web pages that were essentially duplicate content aside from city, region, or state names. Then, as now, SEOs often created web pages for search engines by excessively repeating keywords within titles, meta descriptions, headings, internal anchor text, and within the content to improve rankings.
Section 4.6 of the research paper explains:
“Some search engines give higher weight to pages containing the query keywords several times. For example, for a given query term, a page that contains it ten times may be higher ranked than a page that contains it only once. To take advantage of such engines, some spam pages replicate their content several times in an attempt to rank higher.”
The research paper explains that search engines compress web pages and use the compressed version to reference the original web page. They note that excessive amounts of redundant words results in a higher level of compressibility. So they set about testing if there’s a correlation between a high level of compressibility and spam.
They write:
“Our approach in this section to locating redundant content within a page is to compress the page; to save space and disk time, search engines often compress web pages after indexing them, but before adding them to a page cache.
…We measure the redundancy of web pages by the compression ratio, the size of the uncompressed page divided by the size of the compressed page. We used GZIP …to compress pages, a fast and effective compression algorithm.”
High Compressibility Correlates To Spam
The results of the research showed that web pages with at least a compression ratio of 4.0 tended to be low quality web pages, spam. However, the highest rates of compressibility became less consistent because there were fewer data points, making it harder to interpret.
Figure 9: Prevalence of spam relative to compressibility of page.
The researchers concluded:
“70% of all sampled pages with a compression ratio of at least 4.0 were judged to be spam.”
But they also discovered that using the compression ratio by itself still resulted in false positives, where non-spam pages were incorrectly identified as spam:
“The compression ratio heuristic described in Section 4.6 fared best, correctly identifying 660 (27.9%) of the spam pages in our collection, while misidentifying 2, 068 (12.0%) of all judged pages.
Using all of the aforementioned features, the classification accuracy after the ten-fold cross validation process is encouraging:
95.4% of our judged pages were classified correctly, while 4.6% were classified incorrectly.
More specifically, for the spam class 1, 940 out of the 2, 364 pages, were classified correctly. For the non-spam class, 14, 440 out of the 14,804 pages were classified correctly. Consequently, 788 pages were classified incorrectly.”
The next section describes an interesting discovery about how to increase the accuracy of using on-page signals for identifying spam.
Insight Into Quality Rankings
The research paper examined multiple on-page signals, including compressibility. They discovered that each individual signal (classifier) was able to find some spam but that relying on any one signal on its own resulted in flagging non-spam pages for spam, which are commonly referred to as false positive.
The researchers made an important discovery that everyone interested in SEO should know, which is that using multiple classifiers increased the accuracy of detecting spam and decreased the likelihood of false positives. Just as important, the compressibility signal only identifies one kind of spam but not the full range of spam.
The takeaway is that compressibility is a good way to identify one kind of spam but there are other kinds of spam that aren’t caught with this one signal. Other kinds of spam were not caught with the compressibility signal.
This is the part that every SEO and publisher should be aware of:
“In the previous section, we presented a number of heuristics for assaying spam web pages. That is, we measured several characteristics of web pages, and found ranges of those characteristics which correlated with a page being spam. Nevertheless, when used individually, no technique uncovers most of the spam in our data set without flagging many non-spam pages as spam.
For example, considering the compression ratio heuristic described in Section 4.6, one of our most promising methods, the average probability of spam for ratios of 4.2 and higher is 72%. But only about 1.5% of all pages fall in this range. This number is far below the 13.8% of spam pages that we identified in our data set.”
So, even though compressibility was one of the better signals for identifying spam, it still was unable to uncover the full range of spam within the dataset the researchers used to test the signals.
Combining Multiple Signals
The above results indicated that individual signals of low quality are less accurate. So they tested using multiple signals. What they discovered was that combining multiple on-page signals for detecting spam resulted in a better accuracy rate with less pages misclassified as spam.
The researchers explained that they tested the use of multiple signals:
“One way of combining our heuristic methods is to view the spam detection problem as a classification problem. In this case, we want to create a classification model (or classifier) which, given a web page, will use the page’s features jointly in order to (correctly, we hope) classify it in one of two classes: spam and non-spam.”
These are their conclusions about using multiple signals:
“We have studied various aspects of content-based spam on the web using a real-world data set from the MSNSearch crawler. We have presented a number of heuristic methods for detecting content based spam. Some of our spam detection methods are more effective than others, however when used in isolation our methods may not identify all of the spam pages. For this reason, we combined our spam-detection methods to create a highly accurate C4.5 classifier. Our classifier can correctly identify 86.2% of all spam pages, while flagging very few legitimate pages as spam.”
Key Insight:
Misidentifying “very few legitimate pages as spam” was a significant breakthrough. The important insight that everyone involved with SEO should take away from this is that one signal by itself can result in false positives. Using multiple signals increases the accuracy.
What this means is that SEO tests of isolated ranking or quality signals will not yield reliable results that can be trusted for making strategy or business decisions.
Takeaways
We don’t know for certain if compressibility is used at the search engines but it’s an easy to use signal that combined with others could be used to catch simple kinds of spam like thousands of city name doorway pages with similar content. Yet even if the search engines don’t use this signal, it does show how easy it is to catch that kind of search engine manipulation and that it’s something search engines are well able to handle today.
Here are the key points of this article to keep in mind:
- Doorway pages with duplicate content is easy to catch because they compress at a higher ratio than normal web pages.
- Groups of web pages with a compression ratio above 4.0 were predominantly spam.
- Negative quality signals used by themselves to catch spam can lead to false positives.
- In this particular test, they discovered that on-page negative quality signals only catch specific types of spam.
- When used alone, the compressibility signal only catches redundancy-type spam, fails to detect other forms of spam, and leads to false positives.
- Combing quality signals improves spam detection accuracy and reduces false positives.
- Search engines today have a higher accuracy of spam detection with the use of AI like Spam Brain.
Read the research paper, which is linked from the Google Scholar page of Marc Najork:
Detecting spam web pages through content analysis
Featured Image by Shutterstock/pathdoc
SEO
New Google Trends SEO Documentation
Google Search Central published new documentation on Google Trends, explaining how to use it for search marketing. This guide serves as an easy to understand introduction for newcomers and a helpful refresher for experienced search marketers and publishers.
The new guide has six sections:
- About Google Trends
- Tutorial on monitoring trends
- How to do keyword research with the tool
- How to prioritize content with Trends data
- How to use Google Trends for competitor research
- How to use Google Trends for analyzing brand awareness and sentiment
The section about monitoring trends advises there are two kinds of rising trends, general and specific trends, which can be useful for developing content to publish on a site.
Using the Explore tool, you can leave the search box empty and view the current rising trends worldwide or use a drop down menu to focus on trends in a specific country. Users can further filter rising trends by time periods, categories and the type of search. The results show rising trends by topic and by keywords.
To search for specific trends users just need to enter the specific queries and then filter them by country, time, categories and type of search.
The section called Content Calendar describes how to use Google Trends to understand which content topics to prioritize.
Google explains:
“Google Trends can be helpful not only to get ideas on what to write, but also to prioritize when to publish it. To help you better prioritize which topics to focus on, try to find seasonal trends in the data. With that information, you can plan ahead to have high quality content available on your site a little before people are searching for it, so that when they do, your content is ready for them.”
Read the new Google Trends documentation:
Get started with Google Trends
Featured Image by Shutterstock/Luis Molinero
You must be logged in to post a comment Login