SEO
6 Things You Can Do to Compete With Big Sites
Zillow, Trulia, Redfin. These names appear in almost every conversation about buying, selling, or renting property. This is not because people are particularly interested in these platforms but because they’ve become the default starting point for most property searches.
The best illustration of this is that out of over 4.5M keywords that Zillow ranks for, bringing them an estimated 32.7M visits from search, the top keyword is “zillow”. And did you know that’s a more popular search term than “houses for sale” or even “apartments”?
You might think there’s really nothing left for realtors and agencies. But here’s the twist: their niche focus is their secret weapon. These local experts can outshine the big names, proving that sometimes, being small is the biggest advantage.
This is where SEO comes in. SEO (search engine optimization) for real estate involves strategies to boost your visibility in Google’s organic search results. This visibility brings free, consistent traffic that grows as you create more optimized content.
The opportunity for boutique, small, and medium real estate businesses lies in four key areas:
- Hyperlocal keyword targeting.
- Long-tail keywords with high intent.
- Local link building.
- Exceptional customer service that fuels positive reviews, boosting your local search rankings.
In other words, you need to do SEO better where it counts.
In this article, I’ll share strategies and tips from SEO experts in the real estate sector, along with insights from high-performing niche sites. Our focus is exclusively on SEO, so we won’t cover search ads or listing your business on aggregators, as you’re likely already doing those.
SEO for real estate faces a few specific challenges. It’s good to know them to understand how to shape your strategy.
Big sites dominate the share of voice. National real estate portals and aggregators often outrank smaller agencies. They’ve got tons of backlinks, tons of well-ranking pages fueled by inventory from practically every possible source, and they are well-optimized for Google. It just so happens that all of that is called authority, which Google likes to promote in search engine result pages (SERPs).
A huge challenge is figuring out where big competitors leave content gaps and missed keyword opportunities. Big real estate platforms dominate the market, so you need to dig deep into what they aren’t addressing.
Both local and national competition. Big sites will appear in both national and local search results. Moreover, chances are on the local level, you’ll be competing with local players who already started investing in SEO.
Our biggest SEO challenge is standing out in local searches amidst fierce competition because we are battling local real estate investors and also national companies.
Real estate SEO is incredibly local. Unlike other industries, where a broad audience can be targeted, real estate businesses must rank well in specific cities or neighborhoods. This means you’re not just competing with the big-name RE platforms but also with other local agencies, making it even harder to stand out.
Serving both sides of the market. As a real estate agent, you’re practically a one-person marketplace serving both sellers and buyers.
Each agent has different ambitions, so they need to ensure their SEO strategy aligns with their overall business goals.
Many topics within real estate will count as Your Money or Your Life (YMYL). In recent years, Google has recognized that certain subjects, including real estate, require higher standards of trustworthiness.
Any content should go through multiple fact checks before publication, and each data point should be well sourced with an external link where possible as this will aid authority.
Now, let’s see what we can do about those challenges.
A well-optimized Google Business Profile (GBP) is crucial to outrank aggregators and local competition. As you probably already know, this free listing appears in Google Search and Maps.
I won’t go into the basics of GBP profiles. I’m sure most of you already have one, and if not, you can get up to speed with our full guide for beginners.
What I’d like to emphasize here is two things.
A GBP is one of your best bets to outrank both big sites and local competitors. I can’t even cite a specific expert here because they’ve all said the same thing. That’s because the so-called map pack featuring GBPs often shows on top of regular organic results.
A GBP is your answer to big brands. They have the marketing budgets, the authority on Google, and brand awareness. A GBP gives you a strong local presence backed by reviews and the effort you take to make the profile stand out.
You have to keep in mind that the profile is not just something that people will see only once, and only if they find it through Google. Even if they discover you in other ways, they will circle back to the GBP to see if you can be trusted.
Secondly, there are a few things that can make or break a GBP:
- Listing each branch separately.
- Giving people reasons to leave a positive review.
- Showing who you are and how you work in the photos feed.
Make sure you list branches separately. This is important because Google ranks GBPs based on the distance of the searcher or the location used in the query to the business (among a few other things). So if you want to be visible in all of the cities or neighborhoods where you have a physical address, make sure to list them separately.
Reviews are one of the most impactful ranking factors for GBPs. Virtually everything about them counts: how many are there, what’s the overall ranking, are they fresh, do you respond to them, etc.. Google pretty much reads them just as a potential client would.
Obviously, the goal is to get as many positive reviews as possible. But here’s the tip: not all of them need to come from actual real estate transactions. You can receive excellent reviews by just being helpful.
Next, list ALL your services. By listing all services, you increase your chances of appearing in a wider range of relevant searches. Example below:
Finally, consider adding photos of your team and client interactions. This isn’t about Google rankings — it’s all about how people think. Photos of your team and happy customers help new clients feel like they know who they’re trusting with their biggest assets. Most GBPs just show normal real estate photos. Just make sure to ask for their permission first.
I’ve gathered a few examples of photo feeds that stood out in my research. Photos like these draw attention yet don’t require much effort to make.
Keywords are the words and phrases that people type into search engines to find what they’re looking for. In SEO, you use keywords as topics for your content so that when someone uses the keywords, they can find your content.
The keyword strategy should focus on niching down if you’re a small or medium-sized real estate business (or you’re working for such a client). Keywords with high search volume are usually harder to rank for. Plus, these big keywords often relate to the national market, not your local area. They’re less likely to bring you leads from nearby customers.
Use the niche market to your advantage and focus on using long-tail keywords with low to medium competition. Rather than looking to target broader terms like “real estate” or “investment property UK”, target more specific phrases like “luxury homes in Manchester” or “affordable property in york”.
A huge challenge is figuring out where big competitors leave content gaps and missed keyword opportunities. Big real estate platforms dominate the market, so you need to dig deep into what they aren’t addressing.
Tools like Ahrefs’ Keywords Explorer show what words people use when looking for real estate to buy or an agent to help them sell. Let’s look at how you can use this tool to find the best types of keywords.
Local and hyperlocal real estate keywords
Local and hyperlocal keywords are search terms that are highly specific to a particular geographic location or small community. These keywords typically include:
- Neighborhood names.
- Street names.
- Local landmarks.
- Local attractions.
- Zip codes or postal codes.
- Specific districts within a city.
- Names of local businesses or institutions.
- City comparison (e.g., Portland vs. Austin).
Rather than targeting neighborhood key terms alone, you should also hit landmarks, popular streets, and more. Build expertise and authority through neighborhood-specific landing pages with unique local content. You sacrifice some volume, but you attract highly qualified traffic and increase your chances of showing up at the top of the right search results pages.
To find your keywords in Keywords Explorer:
- Type in broad terms related to the area you operate. For example, in Las Vegas that could be “las vegas, arts district, charleston heights, fremont street, green valley, henderson, los prados, sumerlin, twin lakes, unlv”.
- Go to the Matching terms report.
- In the Include filter add types of the real estate you offer. For example “real estate, house, condo, homes, properties”. Make sure to use the Any mode.
From that point, you can use additional filters to refine results. For example, to find low to medium-difficulty keywords set the KD filter to Max 30.
As you browse through the keywords, add them to a list.
Here are some examples I found:
Questions and real estate buying/selling terminology
Answers to popular questions and terminology allow you to attract customers seeking information first, show off your relevant listings, and get people to contact you for more details.
You can use these same seed keywords to find questions that buyers and sellers are asking. All you need to do is use the Questions tab:
For example, here’s a page explaining some of the basic terms. It generates an estimated 795 organic visits each month.
Local guides
These keywords include a geographic location and offer insights about the local area, like neighborhoods, restaurants, bars, attractions, or real estate market trends.
For instance, Live Love Santa Cruz, a boutique real estate, targets various keywords related to local services and attractions. I’ve listed some keywords where she ranks in the top 10: sushi, beaches, surfboard services; you get the idea. It’s practically a local guide’s blog attached to a real estate business.
To find these keywords, you can again use the standard set of locations and these modifier keywords: “best, things, top-rated, event*, guide, list, tips, map, information, resource, transportation, park*, recreation, shopping”. You can add your own or ask AI to expand this list.
Sidenote.
The asterisk acts as a wildcard for modifier keywords. It will automatically include all the words that start with “park.”
Since these keywords can have irregular structures, it’s a good idea to use competitive keyword research.
To do this, type in “blogs about [local area]” or “[local area] blogs” in Google. If you’re an Ahrefs user, you can use the toolbar to reveal SEO data for each site. Choose sites with the most traffic (ST) and click the KW link.
This will show you the keywords the site ranks for — your new source of content inspiration.
Unique features and buying scenarios
Brandy Hastings from SmartSites and Ally Dyck from seoplus+ mentioned a specific subset of keywords: properties with unique features and specific buying scenarios. For instance “pet-friendly apartments in [suburb]” or “townhomes for sale with low HOA fees”. These keywords typically have low search volume, but they’re high in intent.
Here are some of the ideas you can use for keyword modifiers: “for, near, with, buyer, close to, invest, relocate, retire, in”. Use them with seed keywords related to the type of realty you offer.
- Type in types of real estate as seed keywords.
- Open Matching terms report.
- Set the following filters. Include: add the modifier words mentioned above. Optional filters for finding easier keywords: KD up to 30, Lowest DR Up to 40 in top 10.
- Open the Cluster by terms tab.
From there, look for the areas you serve and browse keyword ideas.
Don’t expect to get leads from every organic visit — it’s a very important thing to understand with this source of traffic.
Real estate decisions take time, and users are often at different stages of their journey. Your goal should be to engage visitors and guide them toward taking the logical next step rather than pushing for an immediate conversion.
I like to think holistically about the different stages someone may be at as they’re researching an area or neighborhood. Informational guides can be really useful for the earlier stages as people are just learning about a place and determining if it’s a good fit. Things like neighborhood overviews, school profiles, guides to local amenities. Then as people start narrowing down their search, more transactional pages optimized for queries like “homes for sale in Neighborhood X” can be effective.
Here’s a simple example of this concept. One of the pages that generates the most traffic for Hudsons Property is a guide to renting and buying a home in London. Each guide links to other relevant content on the site, including London areas.
The visitor can learn not only how to buy or rent but also where. The area guides take them a step further in their buyer’s journey, providing a form to inquire about real estate options.
And that is the whole idea. Each page needs to deliver a logical next step for the visitor to get in touch.
Here are some other ways real estate sites try to engage visitors.
Highlight selected real estate in a neighborhood guide. Even if someone is not ready to buy yet, pictures of nice homes will likely draw them in.
Get Calendly and integrate it with your site. This will give people a quick and easy way to contact you, without back-and-forth with setting up meeting dates.
Encourage contact by making a special offer, such as a free valuation.
Keep main contact options visible at all times. You can include them in a floating menu bar. Simple yet effective. It reduces the time to find contact details and demonstrates your openness.
Within seconds, visitors form lasting impressions about your credibility and professionalism. If they feel something is off, they will leave.
Establishing trust isn’t just about appealing to human psychology — it’s also a critical factor in Google’s ranking algorithm, built into the EEAT concept.
EEAT is how Google’s systems are trained to determine a page’s credibility. The acronym stands for Experience, Expertise, Authoritativeness, and Trustworthiness, with the last element being the most important.
It basically means that a website exhibiting strong EEAT signals is more likely to rank well in search results because Google aims to provide users with credible and reliable information.
Here are some ways you can cater to potential customers and Google.
Getting a TLS certificate is an absolute must. This protects sensitive information, like login credentials and personal details, from being intercepted by malicious actors. It also displays a padlock icon in the browser’s address bar, visually signaling to users that the website is secure.
On Dana Fitzpatrick’s site, I found these few hundred pixels that clearly establish this realtor’s credibility. It features impressive performance data, a compelling testimonial, and a series of recognitions highlighting her experience.
On Nathan Sherman’s site, I found this:
And I couldn’t agree more — these sold properties acts as strong testimonials. They’re not just a list of past transactions; they’re a visual showcase of an agency’s success story.
Moreover, they catch your eye, because not every agency keeps their sold properties in a visible spot on their site. I know it caught my attention when I first saw this after looking at dozens of real estate sites.
Here’s a real estate business that went the extra mile, although with very simple means. Room Real Estate captured the family business spirit in a short video. This video introduces the visitor to two generations who have worked hard for their success.
There’s going to be a lot of visual content on your site, so make sure the images are compressed and the code is optimized. This will keep your site fast which, again, matters both to visitors and Google.
Real estate websites often rely heavily on images and virtual tours, which can slow down site speed if not optimized properly. In one case, we improved a client’s site speed by compressing images and restructuring their code, which led to a significant boost in their search rankings.
Mobile-friendliness is a ranking factor and a must-have if you want your visitors to stick around. Photos of houses and apartments look better on a big screen, but many of your visitors will use a smartphone instead.
To illustrate, here’s the mobile vs desktop distribution looks on most real estate-related keywords I’ve seen so far.
It’s very easy to test your site for these factors. You can use free tools such as our Webmaster Tools. The tool will show you affected pages, tips on how to fix them, and whether the changes you implemented worked.
That said, fixing those things will require some technical skills. So unless you’re a realtor by day and a web developer by night, you might want to get someone to help you.
Backlinks, also known as inbound or incoming links, are links from one website to another. Search engines like Google see these links as votes of trust.
Link building is one of the pillars of SEO as more backlinks from unique domains can improve your search rankings.
There are many tactics to get backlinks, so you need to choose wisely. Based on expert opinions and an analysis of high-performing real estate sites, here’s where you can get quality backlinks:
- Directories.
- Press.
- Podcasts, shows, and public speaking.
- Local organizations, schols, and events.
- Your terminology and data pages.
Let’s look at them in more detail.
1. Directories
Directories are organized listings of websites, typically categorized by topic, industry, or location. For example, Circa is a niche directory for old house listings. They also feature agents and brokers.
Getting your site on a directory is pretty simple. Depending on where you’re listing, you might just add your info yourself, fill out a form and wait for approval, or “pay to play”.
A quick search of online directories or business listings will give you enough sites to keep you busy for a few hours (for example this list from HubSpot). On top of that, I’d recommend you also check out our advanced guide to this type of link building and find some hidden gems.
2. Press
Backlinks from the press come from providing journalists and bloggers a reason to mention you, and therefore, link to you.
For instance, you can offer expertise like Michael Bondi.
Or get your listings featured like Berkshire Property Agents.
You will find lots of requests from journalists requests on HARO, Help a B2B Writer, and similar sites.
Consider reaching out to local press outlets with real estate-focused story ideas. For example, you could propose an article exploring ‘Why there’s a surge of homes for sale in [specific area]’. Alternatively, offer your expertise to journalists working on real estate-related pieces.
If you have a bit more budget, you can hire a PR or link building agency to seek out the right opportunities.
3. Podcasts, shows and public speaking
These events often list speakers or participants on their websites, providing an opportunity for valuable backlinks from reputable sources.
Whenever you get a chance to appear on a show, conference, or lecture, ask for a link back to your site.
4. Local organizations, schools and events
Local organizations, schools, and events often link to sponsors, businesses that are involved in charities or community initiatives, and helpful resources.
These backlinks might need a bit more effort but the benefits of networking will likely surpass SEO. Here are a few ideas to try:
- Join local business associations and chambers of commerce.
- Reach out to local schools and offer to participate in career days or provide educational resources about real estate.
- Sponsor local sports teams or cultural events.
- Volunteer for community service projects or organize charity events.
- Create and share valuable content about the local real estate market, homebuying tips, or neighborhood guides.
- Offer free workshops or seminars on real estate topics for community members.
- Partner with local non-profits for fundraising initiatives.
- Offer internship opportunities to local students interested in real estate.
5. Terminology and data pages
Citing data and facts is one of the most popular reasons to link. Become the source, and you might earn lots of links this way passively.
To get an idea of what kind of resources earn links in real estate, you can look at competitors’ sites in Ahrefs’ Site Explorer. Just paste their domain and go to the Best by links report.
Tip
Before investing time in link building, I strongly encourage you to read our beginner’s guide. Learn how to tell good links from links that are less likely to give you a boost, and which practices could possibly hurt your site.
Final thoughts
I’d like to leave you with two more tips.
I’ve seen many realtors create video content for YouTube, including virtual property tours, and neighborhood showcases. However, just a few of my sources mentioned this strategy. For inspiration, check out Brad McCallum’s channel. To find keywords for YouTube SEO, you can use tools like vidIQ.
Finally, I want to quickly discuss your KPIs. Since buyer’s journey in this business can be quite long, a good idea would be to track the correlation between SEO metrics and closed deals. To crunch the numbers, simply ask ChatGPT.
We measure the ROI of SEO from the number of quality leads that are generated by our website and then correlate them with closed deals, giving a clear picture of how organic search is contributing to our bottom line.
Got questions or comments? Send me a message on LinkedIn.
SEO
How to Revive an Old Blog Article for SEO
Quick question: What do you typically do with your old blog posts? Most likely, the answer is: Not much.
If that’s the case, you’re not alone. Many of us in SEO and content marketing tend to focus on continuously creating new content, rather than leveraging our existing blog posts.
However, here’s the reality—Google is becoming increasingly sophisticated in evaluating content quality, and we need to adapt accordingly. Just as it’s easier to encourage existing customers to make repeat purchases, updating old content on your website is a more efficient and sustainable strategy in the long run.
Ways to Optimize Older Content
Some of your old content might not be optimized for SEO very well, rank for irrelevant keywords, or drive no traffic at all. If the quality is still decent, however, you should be able to optimize it properly with little effort.
Refresh Content
If your blog post contains a specific year or mentions current events, it may become outdated over time. If the rest of the content is still relevant (like if it’s targeting an evergreen topic), simply updating the date might be all you need to do.
Rewrite Old Blog Posts
When the content quality is low (you might have greatly improved your writing skills since you’ve written the post) but the potential is still there, there’s not much you can do apart from rewriting an old blog post completely.
This is not a waste—you’re saving time on brainstorming since the basic structure is already in place. Now, focus on improving the quality.
Delete Old Blog Posts
You might find a blog post that just seems unusable. Should you delete your old content? It depends. If it’s completely outdated, of low quality, and irrelevant to any valuable keywords for your website, it’s better to remove it.
Once you decide to delete the post, don’t forget to set up a 301 redirect to a related post or page, or to your homepage.
Promote Old Blog Posts
Sometimes all your content needs is a bit of promotion to start ranking and getting traffic again. Share it on your social media, link to it from a new post – do something to get it discoverable again to your audience. This can give it the boost it needs to attract organic links too.
Which Blog Posts Should You Update?
Deciding when to update or rewrite blog posts is a decision that relies on one important thing: a content audit.
Use your Google Analytics to find out which blog posts used to drive tons of traffic, but no longer have the same reach. You can also use Google Search Console to find out which of your blog posts have lost visibility in comparison to previous months. I have a guide on website analysis using Google Analytics and Google Search Console you can follow.
If you use keyword tracking tools like SE Ranking, you can also use the data it provides to come up with a list of blog posts that have dropped in the rankings.
Make data-driven decisions to identify which blog posts would benefit from these updates – i.e., which ones still have the chance to recover their keyword rankings and organic traffic.
With Google’s helpful content update, which emphasizes better user experiences, it’s crucial to ensure your content remains relevant, valuable, and up-to-date.
How To Update Old Blog Posts for SEO
Updating articles can be an involved process. Here are some tips and tactics to help you get it right.
Author’s Note: I have a Comprehensive On-Page SEO Checklist you might also be interested in following while you’re doing your content audit.
Conduct New Keyword Research
Updating your post without any guide won’t get you far. Always do your keyword research to understand how users are searching for your given topic.
Proper research can also show you relevant questions and sections that can be added to the blog post you’re updating or rewriting. Make sure to take a look at the People Also Ask (PAA) section that shows up when you search for your target keyword. Check out other websites like Answer The Public, Reddit, and Quora to see what users are looking for too.
Look for New Ranking Opportunities
When trying to revive an old blog post for SEO, keep an eye out for new SEO opportunities (e.g., AI Overview, featured snippets, and related search terms) that didn’t exist when you first wrote your blog post. Some of these features can be targeted by the new content you will add to your post, if you write with the aim to be eligible for it.
Rewrite Headlines and Meta Tags
If you want to attract new readers, consider updating your headlines and meta tags.
Your headlines and meta tags should fulfill these three things:
- Reflect the rewritten and new content you’ve added to the blog post.
- Be optimized for the new keywords it’s targeting (if any).
- Appeal to your target audience – who may have changed tastes from when the blog post was originally made.
Remember that your meta tags in particular act like a brief advertisement for your blog post, since this is what the user first sees when your blog post is shown in the search results page.
Take a look at your blog post’s click-through rate on Google Search Console – if it falls below 2%, it’s definitely time for new meta tags.
Replace Outdated Information and Statistics
Updating blog content with current studies and statistics enhances the relevance and credibility of your post. By providing up-to-date information, you help your audience make better, well-informed decisions, while also showing that your content is trustworthy.
Tighten or Expand Ideas
Your old content might be too short to provide real value to users – or you might have rambled on and on in your post. It’s important to evaluate whether you need to make your content more concise, or if you need to elaborate more.
Keep the following tips in mind as you refine your blog post’s ideas:
- Evaluate Helpfulness: Measure how well your content addresses your readers’ pain points. Aim to follow the E-E-A-T model (Experience, Expertise, Authoritativeness, Trustworthiness).
- Identify Missing Context: Consider whether your content needs more detail or clarification. View it from your audience’s perspective and ask if the information is complete, or if more information is needed.
- Interview Experts: Speak with industry experts or thought leaders to get fresh insights. This will help support your writing, and provide unique points that enhance the value of your content.
- Use Better Examples: Examples help simplify complex concepts. Add new examples or improve existing ones to strengthen your points.
- Add New Sections if Needed: If your content lacks depth or misses a key point, add new sections to cover these areas more thoroughly.
- Remove Fluff: Every sentence should contribute to the overall narrative. Eliminate unnecessary content to make your post more concise.
- Revise Listicles: Update listicle items based on SEO recommendations and content quality. Add or remove headings to stay competitive with higher-ranking posts.
Improve Visuals and Other Media
No doubt that there are tons of old graphics and photos in your blog posts that can be improved with the tools we have today. Make sure all of the visuals used in your content are appealing and high quality.
Update Internal and External Links
Are your internal and external links up to date? They need to be for your SEO and user experience. Outdated links can lead to broken pages or irrelevant content, frustrating readers and hurting your site’s performance.
You need to check for any broken links on your old blog posts, and update them ASAP. Updating your old blog posts can also lead to new opportunities to link internally to other blog posts and pages, which may not have been available when the post was originally published.
Optimize for Conversions
When updating content, the ultimate goal is often to increase conversions. However, your conversion goals may have changed over the years.
So here’s what you need to check in your updated blog post. First, does the call-to-action (CTA) still link to the products or services you want to promote? If not, update it to direct readers to the current solution or offer.
Second, consider where you can use different conversion strategies. Don’t just add a CTA at the end of the post.
Last, make sure that the blog post leverages product-led content. It’s going to help you mention your products and services in a way that feels natural, without being too pushy. Being subtle can be a high ROI tactic for updated posts.
Key Takeaway
Reviving old blog articles for SEO is a powerful strategy that can breathe new life into your content and boost your website’s visibility. Instead of solely focusing on creating new posts, taking the time to refresh existing content can yield impressive results, both in terms of traffic and conversions.
By implementing these strategies, you can transform old blog posts into valuable resources that attract new readers and retain existing ones. So, roll up your sleeves, dive into your archives, and start updating your content today—your audience and search rankings will thank you!
SEO
How Compression Can Be Used To Detect Low Quality Pages
The concept of Compressibility as a quality signal is not widely known, but SEOs should be aware of it. Search engines can use web page compressibility to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords, making it useful knowledge for SEO.
Although the following research paper demonstrates a successful use of on-page features for detecting spam, the deliberate lack of transparency by search engines makes it difficult to say with certainty if search engines are applying this or similar techniques.
What Is Compressibility?
In computing, compressibility refers to how much a file (data) can be reduced in size while retaining essential information, typically to maximize storage space or to allow more data to be transmitted over the Internet.
TL/DR Of Compression
Compression replaces repeated words and phrases with shorter references, reducing the file size by significant margins. Search engines typically compress indexed web pages to maximize storage space, reduce bandwidth, and improve retrieval speed, among other reasons.
This is a simplified explanation of how compression works:
- Identify Patterns:
A compression algorithm scans the text to find repeated words, patterns and phrases - Shorter Codes Take Up Less Space:
The codes and symbols use less storage space then the original words and phrases, which results in a smaller file size. - Shorter References Use Less Bits:
The “code” that essentially symbolizes the replaced words and phrases uses less data than the originals.
A bonus effect of using compression is that it can also be used to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords.
Research Paper About Detecting Spam
This research paper is significant because it was authored by distinguished computer scientists known for breakthroughs in AI, distributed computing, information retrieval, and other fields.
Marc Najork
One of the co-authors of the research paper is Marc Najork, a prominent research scientist who currently holds the title of Distinguished Research Scientist at Google DeepMind. He’s a co-author of the papers for TW-BERT, has contributed research for increasing the accuracy of using implicit user feedback like clicks, and worked on creating improved AI-based information retrieval (DSI++: Updating Transformer Memory with New Documents), among many other major breakthroughs in information retrieval.
Dennis Fetterly
Another of the co-authors is Dennis Fetterly, currently a software engineer at Google. He is listed as a co-inventor in a patent for a ranking algorithm that uses links, and is known for his research in distributed computing and information retrieval.
Those are just two of the distinguished researchers listed as co-authors of the 2006 Microsoft research paper about identifying spam through on-page content features. Among the several on-page content features the research paper analyzes is compressibility, which they discovered can be used as a classifier for indicating that a web page is spammy.
Detecting Spam Web Pages Through Content Analysis
Although the research paper was authored in 2006, its findings remain relevant to today.
Then, as now, people attempted to rank hundreds or thousands of location-based web pages that were essentially duplicate content aside from city, region, or state names. Then, as now, SEOs often created web pages for search engines by excessively repeating keywords within titles, meta descriptions, headings, internal anchor text, and within the content to improve rankings.
Section 4.6 of the research paper explains:
“Some search engines give higher weight to pages containing the query keywords several times. For example, for a given query term, a page that contains it ten times may be higher ranked than a page that contains it only once. To take advantage of such engines, some spam pages replicate their content several times in an attempt to rank higher.”
The research paper explains that search engines compress web pages and use the compressed version to reference the original web page. They note that excessive amounts of redundant words results in a higher level of compressibility. So they set about testing if there’s a correlation between a high level of compressibility and spam.
They write:
“Our approach in this section to locating redundant content within a page is to compress the page; to save space and disk time, search engines often compress web pages after indexing them, but before adding them to a page cache.
…We measure the redundancy of web pages by the compression ratio, the size of the uncompressed page divided by the size of the compressed page. We used GZIP …to compress pages, a fast and effective compression algorithm.”
High Compressibility Correlates To Spam
The results of the research showed that web pages with at least a compression ratio of 4.0 tended to be low quality web pages, spam. However, the highest rates of compressibility became less consistent because there were fewer data points, making it harder to interpret.
Figure 9: Prevalence of spam relative to compressibility of page.
The researchers concluded:
“70% of all sampled pages with a compression ratio of at least 4.0 were judged to be spam.”
But they also discovered that using the compression ratio by itself still resulted in false positives, where non-spam pages were incorrectly identified as spam:
“The compression ratio heuristic described in Section 4.6 fared best, correctly identifying 660 (27.9%) of the spam pages in our collection, while misidentifying 2, 068 (12.0%) of all judged pages.
Using all of the aforementioned features, the classification accuracy after the ten-fold cross validation process is encouraging:
95.4% of our judged pages were classified correctly, while 4.6% were classified incorrectly.
More specifically, for the spam class 1, 940 out of the 2, 364 pages, were classified correctly. For the non-spam class, 14, 440 out of the 14,804 pages were classified correctly. Consequently, 788 pages were classified incorrectly.”
The next section describes an interesting discovery about how to increase the accuracy of using on-page signals for identifying spam.
Insight Into Quality Rankings
The research paper examined multiple on-page signals, including compressibility. They discovered that each individual signal (classifier) was able to find some spam but that relying on any one signal on its own resulted in flagging non-spam pages for spam, which are commonly referred to as false positive.
The researchers made an important discovery that everyone interested in SEO should know, which is that using multiple classifiers increased the accuracy of detecting spam and decreased the likelihood of false positives. Just as important, the compressibility signal only identifies one kind of spam but not the full range of spam.
The takeaway is that compressibility is a good way to identify one kind of spam but there are other kinds of spam that aren’t caught with this one signal. Other kinds of spam were not caught with the compressibility signal.
This is the part that every SEO and publisher should be aware of:
“In the previous section, we presented a number of heuristics for assaying spam web pages. That is, we measured several characteristics of web pages, and found ranges of those characteristics which correlated with a page being spam. Nevertheless, when used individually, no technique uncovers most of the spam in our data set without flagging many non-spam pages as spam.
For example, considering the compression ratio heuristic described in Section 4.6, one of our most promising methods, the average probability of spam for ratios of 4.2 and higher is 72%. But only about 1.5% of all pages fall in this range. This number is far below the 13.8% of spam pages that we identified in our data set.”
So, even though compressibility was one of the better signals for identifying spam, it still was unable to uncover the full range of spam within the dataset the researchers used to test the signals.
Combining Multiple Signals
The above results indicated that individual signals of low quality are less accurate. So they tested using multiple signals. What they discovered was that combining multiple on-page signals for detecting spam resulted in a better accuracy rate with less pages misclassified as spam.
The researchers explained that they tested the use of multiple signals:
“One way of combining our heuristic methods is to view the spam detection problem as a classification problem. In this case, we want to create a classification model (or classifier) which, given a web page, will use the page’s features jointly in order to (correctly, we hope) classify it in one of two classes: spam and non-spam.”
These are their conclusions about using multiple signals:
“We have studied various aspects of content-based spam on the web using a real-world data set from the MSNSearch crawler. We have presented a number of heuristic methods for detecting content based spam. Some of our spam detection methods are more effective than others, however when used in isolation our methods may not identify all of the spam pages. For this reason, we combined our spam-detection methods to create a highly accurate C4.5 classifier. Our classifier can correctly identify 86.2% of all spam pages, while flagging very few legitimate pages as spam.”
Key Insight:
Misidentifying “very few legitimate pages as spam” was a significant breakthrough. The important insight that everyone involved with SEO should take away from this is that one signal by itself can result in false positives. Using multiple signals increases the accuracy.
What this means is that SEO tests of isolated ranking or quality signals will not yield reliable results that can be trusted for making strategy or business decisions.
Takeaways
We don’t know for certain if compressibility is used at the search engines but it’s an easy to use signal that combined with others could be used to catch simple kinds of spam like thousands of city name doorway pages with similar content. Yet even if the search engines don’t use this signal, it does show how easy it is to catch that kind of search engine manipulation and that it’s something search engines are well able to handle today.
Here are the key points of this article to keep in mind:
- Doorway pages with duplicate content is easy to catch because they compress at a higher ratio than normal web pages.
- Groups of web pages with a compression ratio above 4.0 were predominantly spam.
- Negative quality signals used by themselves to catch spam can lead to false positives.
- In this particular test, they discovered that on-page negative quality signals only catch specific types of spam.
- When used alone, the compressibility signal only catches redundancy-type spam, fails to detect other forms of spam, and leads to false positives.
- Combing quality signals improves spam detection accuracy and reduces false positives.
- Search engines today have a higher accuracy of spam detection with the use of AI like Spam Brain.
Read the research paper, which is linked from the Google Scholar page of Marc Najork:
Detecting spam web pages through content analysis
Featured Image by Shutterstock/pathdoc
SEO
New Google Trends SEO Documentation
Google Search Central published new documentation on Google Trends, explaining how to use it for search marketing. This guide serves as an easy to understand introduction for newcomers and a helpful refresher for experienced search marketers and publishers.
The new guide has six sections:
- About Google Trends
- Tutorial on monitoring trends
- How to do keyword research with the tool
- How to prioritize content with Trends data
- How to use Google Trends for competitor research
- How to use Google Trends for analyzing brand awareness and sentiment
The section about monitoring trends advises there are two kinds of rising trends, general and specific trends, which can be useful for developing content to publish on a site.
Using the Explore tool, you can leave the search box empty and view the current rising trends worldwide or use a drop down menu to focus on trends in a specific country. Users can further filter rising trends by time periods, categories and the type of search. The results show rising trends by topic and by keywords.
To search for specific trends users just need to enter the specific queries and then filter them by country, time, categories and type of search.
The section called Content Calendar describes how to use Google Trends to understand which content topics to prioritize.
Google explains:
“Google Trends can be helpful not only to get ideas on what to write, but also to prioritize when to publish it. To help you better prioritize which topics to focus on, try to find seasonal trends in the data. With that information, you can plan ahead to have high quality content available on your site a little before people are searching for it, so that when they do, your content is ready for them.”
Read the new Google Trends documentation:
Get started with Google Trends
Featured Image by Shutterstock/Luis Molinero