Connect with us

SEO

Best Practices to Make It SEO-Friendly

Published

on

Best Practices to Make It SEO-Friendly

The increasing prevalence of React in modern web development cannot be ignored.

React and other similar libraries (like Vue.js) are becoming the de facto choice for larger businesses that require complex development where a more simplistic approach (like using a WordPress theme) won’t satisfy the requirements.

Despite that, SEOs did not initially embrace libraries like React due to search engines struggling to effectively render JavaScript, with content available within the HTML source being the preference.

However, developments in both how Google and React can render JavaScript have simplified these complexities, resulting in SEO no longer being the blocker for using React.

Still, some complexities remain, which I’ll run through in this guide.

On that note, here’s what we’ll cover:

Advertisement

But first, what is React?

React is an open-source JavaScript library developed by Meta (formerly Facebook) for building web and mobile applications. The main features of React are that it is declarative, is component-based, and allows easier manipulation of the DOM.

The simplest way to understand the components is by thinking of them as plugins, like for WordPress. They allow developers to quickly build a design and add functionality to a page using component libraries like MUI or Tailwind UI.

If you want the full lowdown on why developers love React, start here:

Advertisement

Rendering with React, a short history

React implements an App Shell Model, meaning the vast majority of content, if not all, will be Client-side Rendered (CSR) by default.

CSR means the HTML primarily contains the React JS library rather than the server sending the entire page’s contents within the initial HTTP response from the server (the HTML source).

It will also include miscellaneous JavaScript containing JSON data or links to JS files that contain React components. You can quickly tell a site is client-side rendered by checking the HTML source. To do that, right-click and select “View Page Source” (or CTRL + U/CMD + U).

Netflix's homepage source HTML

A screenshot of the netlfix.com homepage source HTML.

If you don’t see many lines of HTML there, the application is likely client-side rendering.

Advertisement

However, when you inspect the element by right-clicking and selecting “Inspect element” (or F12/CMD + ⌥ + I), you’ll see the DOM generated by the browser (where the browser has rendered JavaScript).

The result is you’ll then see the site has a lot of HTML:

Lots of HTML

Note the appMountPoint ID on the first <div>. You’ll commonly see an element like that on a single-page application (SPA), so a library like React knows where it should inject HTML. Technology detection tools, e.g., Wappalyzer, are also great at detecting the library.

Editor’s Note

Ahrefs’ Site Audit saves both the Raw HTML sent from the server and the Rendered HTML in the browser, making it easier to spot whether a site has client-side rendered content.

Gif showing Site Audit saves both Raw HTML and Rendered HTML

Even better, you can search both the Raw and Rendered HTML to know what content is specifically being rendered client-side. In the below example, you can see this site is client-side rendering key page content, such as the <h1> tag.

Advertisement

Gif showing site is client-side rendering key page content

Joshua Hardwick

Websites created using React differ from the more traditional approach of leaving the heavy-lifting of rendering content on the server using languages like PHP—called Server-side Rendering (SSR).

Flowchart showing the SSR process

The above shows the server rendering JavaScript into HTML with React (more on that shortly). The concept is the same for sites built with PHP (like WordPress). It’s just PHP being turned into HTML rather than JavaScript.

Before SSR, developers kept it even simpler.

They would create static HTML documents that didn’t change, host them on a server, and then send them immediately. The server didn’t need to render anything, and the browser often had very little to render.

SPAs (including those using React) are now coming full circle back to this static approach. They’re now pre-rendering JavaScript into HTML before a browser requests the URL. This approach is called Static Site Generation (SSG), also known as Static Rendering.

Two flowcharts showing the SSG process

In practice, SSR and SSG are similar.

The key difference is that rendering happens with SSR when a browser requests a URL versus a framework pre-rendering content at build time with SSG (when developers deploy new code or a web admin changes the site’s content).

Advertisement

SSR can be more dynamic but slower due to additional latency while the server renders the content before sending it to the user’s browser.

SSG is faster, as the content has already been rendered, meaning it can be served to the user immediately (meaning a quicker TTFB).

How Google processes pages

To understand why React’s default client-side rendering approach causes SEO issues, you first need to know how Google crawls, processes, and indexes pages.

We can summarize the basics of how this works in the below steps:

Advertisement
  1. Crawling – Googlebot sends GET requests to a server for the URLs in the crawl queue and saves the response contents. Googlebot does this for HTML, JS, CSS, image files, and more.
  2. Processing – This includes adding URLs to the crawl queue found within <a href> links within the HTML. It also includes queuing resource URLs (CSS/JS) found within <link> tags or images within <img src> tags. If Googlebot finds a noindex tag at this stage, the process stops, Googlebot won’t render the content, and Caffeine (Google’s indexer) won’t index it.
  3. Rendering – Googlebot executes JavaScript code with a headless Chromium browser to find additional content within the DOM, but not the HTML source. It does this for all HTML URLs.
  4. Indexing – Caffeine takes the information from Googlebot, normalizes it (fixes broken HTML), and then tries to make sense of it all, precomputing some ranking signals ready for serving within a search result.
Flowchart showing how Google crawls, processes, and indexes pages

Historically, issues with React and other JS libraries have been due to Google not handling the rendering step well.

Some examples include:

  • Not rendering JavaScript – It’s an older issue, but Google only started rendering JavaScript in a limited way in 2008. However, it was still reliant on a crawling scheme for JavaScript sites created in 2009. (Google has since deprecated the scheme.)
  • The rendering engine (Chromium) being out of date – This resulted in a lack of support for the latest browser and JavaScript features. If you used a JavaScript feature that Googlebot didn’t support, your page might not render correctly, which could negatively impact your content’s indexing.
  • Google had a rendering delay – In some cases, this could mean a delay of up to a few weeks, slowing down the time for changes to the content to reach the indexing stage. This would have ruled out relying on Google to render content for most sites.

Thankfully, Google has now resolved most of these issues. Googlebot is now evergreen, meaning it always supports the latest features of Chromium.

In addition, the rendering delay is now five seconds, as announced by Martin Splitt at the Chrome Developer Summit in November 2019:

Last year Tom Greenaway and I were on this stage and telling you, ‘Well, you know, it can take up to a week, we are very sorry for this.’ Forget this, okay? Because the new numbers look a lot better. So we actually went over the numbers and found that, it turns out that at median, the time we spent between crawling and actually having rendered these results is – on median – it’s five seconds!”

This all sounds positive. But is client-side rendering and leaving Googlebot to render content the right strategy?

The answer is most likely still no.

Common SEO issues with React

Advertisement

In the past five years, Google has innovated its handling of JavaScript content, but entirely client-side rendered sites introduce other issues that you need to consider.

It’s important to note that you can overcome all issues with React and SEO.

React JS is a development tool. React is no different from any other tool within a development stack, whether that’s a WordPress plugin or the CDN you choose. How you configure it will decide whether it detracts or enhances SEO.

Ultimately, React is good for SEO, as it improves user experience. You just need to make sure you consider the following common issues.

1. Pick the right rendering strategy

The most significant issue you’ll need to tackle with React is how it renders content.

As mentioned, Google is great at rendering JavaScript nowadays. But unfortunately, that isn’t the case with other search engines. Bing has some support for JavaScript rendering, although its efficiency is unknown. Other search engines like Baidu, Yandex, and others offer limited support.

Advertisement

Sidenote.

This limitation doesn’t only impact search engines. Apart from site auditors, SEO tools that crawl the web and provide critical data on elements like a site’s backlinks do not render JavaScript. This can have a significant impact on the quality of data they provide. The only exception is Ahrefs, which has been rendering JavaScript across the web since 2017 and currently renders over 200 million pages per day.

Introducing this unknown builds a good case for opting for a server-side rendered solution to ensure that all crawlers can see the site’s content.

In addition, rendering content on the server has another crucial benefit: load times.

Load times

Rendering JavaScript is intensive on the CPU; this makes large libraries like React slower to load and become interactive for users. You’ll generally see Core Web Vitals, such as Time to Interactive (TTI), being much higher for SPAs—especially on mobile, the primary way users consume web content.

Overview of metrics' performance, including FCP, LCP, etc

An example React application that utilizes client-side rendering.

However, after the initial render by the browser, subsequent load times tend to be quicker due to the following:

Advertisement

Depending on the number of pages viewed per visit, this can result in field data being positive overall.

Four bar graphs showing positive field data of FCP, LCP, FID, and CLS

However, if your site has a low number of pages viewed per visit, you’ll struggle to get positive field data for all Core Web Vitals.

Solution

The best option is to opt for SSR or SSG mainly due to:

  • Faster initial renders.
  • Not having to rely on search engine crawlers to render content.
  • Improvements in TTI due to less JavaScript code for the browser to parse and render before becoming interactive.

Implementing SSR within React is possible via ReactDOMServer. However, I recommend using a React framework called Next.js and using its SSG and SSR options. You can also implement CSR with Next.js, but the framework nudges users toward SSR/SSG due to speed.

Next.js supports what it calls “Automatic Static Optimization.” In practice, this means you can have some pages on a site that use SSR (such as an account page) and other pages using SSG (such as your blog).

The result: SSG and fast TTFB for non-dynamic pages, and SSR as a backup rendering strategy for dynamic content.

Sidenote.

Advertisement

You may have heard about React Hydration with ReactDOM.hydrate(). This is where content is delivered via SSG/SSR and then turns into a client-side rendered application during the initial render. This may be the obvious choice for dynamic applications in the future rather than SSR. However, hydration currently works by loading the entire React library and then attaching event handlers to HTML that will change. React then keeps HTML between the browser and server in sync. Currently, I can’t recommend this approach because it still has negative implications for web vitals like TTI for the initial render. Partial Hydration may resolve this in the future by only hydrating critical parts of the page (like ones within the browser viewport) rather than the entire page; until then, SSR/SSG is the better option.

Since we’re talking about speed, I’ll be doing you a disservice by not mentioning other ways Next.js optimizes the critical rendering path for React applications with features like:

  • Image optimization – This adds width and height <img> attributes and srcset, lazy loading, and image resizing.
  • Font optimization – This inlines critical font CSS and adds controls for font-display.
  • Script optimization – This lets you pick when a script should be loaded: before/after the page is interactive or lazily.
  • Dynamic imports – If you implement best practices for code splitting, this feature makes it easier to import JS code when required rather than leaving it to load on the initial render and slowing it down.

Speed and positive Core Web Vitals are a ranking factor, albeit a minor one. Next.js features make it easier to create great web experiences that will give you a competitive advantage.

TIP

Many developers deploy their Next.js web applications using Vercel (the creators of Next.js), which has a global edge network of servers; this results in fast load times.

Vercel provides data on the Core Web Vitals of all sites deployed on the platform, but you can also get detailed web vital data for each URL using Ahrefs’ Site Audit.

Simply add an API key within the crawl settings of your projects.

Text field to add API key

After you’ve run your audit, have a look at the performance area. There, Ahrefs’ Site Audit will show you charts displaying data from the Chrome User Experience Report (CrUX) and Lighthouse.

Advertisement
Pie charts and bar graphs showing data from CrUX and Lighthouse

2. Use status codes correctly

A common issue with most SPAs is they don’t correctly report status codes. This is as the server isn’t loading the page—the browser is. You’ll commonly see issues with:

  • No 3xx redirects, with JavaScript redirects being used instead.
  • 4xx status codes not reporting for “not found” URLs.

You can see below I ran a test on a React site with httpstatus.io. This page should obviously be a 404 but, instead, returns a 200 status code. This is called a soft 404.

Table showing URL on left. On right, under "Status codes," it shows "200"

The risk here is that Google may decide to index that page (depending on its content). Google could then serve this to users, or it’ll be used when evaluating a site.

In addition, reporting 404s helps SEOs audit a site. If you accidentally internally link to a 404 page and it’s returning a 200 status code, quickly spotting the area with an auditing tool may become much more challenging.

There are a couple of ways to solve this issue. If you’re client-side rendering:

  1. Use the React Router framework.
  2. Create a 404 component that shows when a route isn’t recognized.
  3. Add a noindex tag to “not found” pages.
  4. Add a <h1> with a message like “404: Page Not Found.” This isn’t ideal, as we don’t report a 404 status code. But it will prevent Google from indexing the page and help it recognize the page as a soft 404.
  5. Use JavaScript redirects when you need to change a URL. Again, not ideal, but Google does follow JavaScript redirects and pass ranking signals.

If you’re using SSR, Next.js makes this simple with response helpers, which let you set whatever status code you want, including 3xx redirects or a 4xx status code. The approach I outlined using React Router can also be put into practice while using Next.js. However, if you’re using Next.js, you’re likely also implementing SSR/SSG.

3. Avoid hashed URLs

This issue isn’t as common for React, but it’s essential to avoid hash URLs like the following:

  • https://reactspa.com/#/shop
  • https://reactspa.com/#/about
  • https://reactspa.com/#/contact

Generally, Google isn’t going to see anything after the hash. All of these pages will be seen as https://reactspa.com/.

Solution

SPAs with client-side routing should implement the History API to change pages.

You can do this relatively easily with both React Router and Next.js.

Advertisement

4. Use <a href> links where relevant

A common mistake with SPAs is using a <div> or a <button> to change the URL. This isn’t an issue with React itself, but how the library is used.

Doing this presents an issue with search engines. As mentioned earlier, when Google processes a URL, it looks for additional URLs to crawl within <a href> elements.

If the <a href> element is missing, Google won’t crawl the URLs and pass PageRank.

Solution

The solution is to include <a href> links to URLs that you want Google to discover.

Checking whether you’re linking to a URL correctly is easy. Inspect the element that internally links and check the HTML to ensure you’ve included <a href> links.

As in the above example, you may have an issue if they aren’t.

Advertisement

However, it’s essential to understand that missing <a href> links aren’t always an issue. One benefit of CSR is that when content is helpful to users but not search engines, you can change the content client-side and not include the <a href> link.

In the above example, the site uses faceted navigation that links to potentially millions of combinations of filters that aren’t useful for a search engine to crawl or index.

List of genres

Loading these filters client-side makes sense here, as the site will conserve crawl budget by not adding <a href> links for Google to crawl.

Next.js makes this easy with its link component, which you can configure to allow client-side navigation.

If you’ve decided to implement a fully CSR application, you can change URLs with React Router using onClick and the History API.

5. Avoid lazy loading essential HTML

It’s common for sites developed with React to inject content into the DOM when a user clicks or hovers over an element—simply because the library makes that easy to do.

Advertisement

This isn’t inherently bad, but content added to the DOM this way will not be seen by search engines. If the content injected includes important textual content or internal links, this may negatively impact:

  • How well the page performs (as Google won’t see the content).
  • The discoverability of other URLs (as Google won’t find the internal links).

Here’s an example on a React JS site I recently audited. Here, I’ll show a well-known e‑commerce brand with important internal links within its faceted navigation.

However, a modal showing the navigation on mobile was injected into the DOM when you clicked a “Filter” button. Watch the second <!—-> within the HTML below to see this in practice:

Gif of modal showing the navigation on mobile was injected into DOM

Solution

Spotting these issues isn’t easy. And as far as I know, no tool will directly tell you about them.

Instead, you should check for common elements such as:

  • Accordions
  • Modals
  • Tabs
  • Mega menus
  • Hamburger menus

You’ll then need to inspect the element on them and watch what happens with the HTML as you open/close them by clicking or hovering (as I have done in the above GIF).

Suppose you notice JavaScript is adding HTML to the page. In that case, you’ll need to work with the developers. This is so that rather than injecting the content into the DOM, it’s included within the HTML by default and is hidden and shown via CSS using properties like visibility: hidden; or display: none;.

6. Don’t forget the fundamentals

While there are additional SEO considerations with React applications, that doesn’t mean other fundamentals don’t apply.

You’ll still need to make sure your React applications follow best practices for:

Advertisement

Final thoughts

Unfortunately, working with React applications does add to the already long list of issues a technical SEO needs to check. But thanks to frameworks like Next.js, it makes the work of an SEO much more straightforward than what it was historically.

Hopefully, this guide has helped you better understand the additional considerations you need to make as an SEO when working with React applications.

Have any questions on working with React? Tweet me.




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

How to Revive an Old Blog Article for SEO

Published

on

Step-by-Step: How to Optimize Old Blog Posts for SEO

Quick question: What do you typically do with your old blog posts? Most likely, the answer is: Not much.

If that’s the case, you’re not alone. Many of us in SEO and content marketing tend to focus on continuously creating new content, rather than leveraging our existing blog posts.

However, here’s the reality—Google is becoming increasingly sophisticated in evaluating content quality, and we need to adapt accordingly. Just as it’s easier to encourage existing customers to make repeat purchases, updating old content on your website is a more efficient and sustainable strategy in the long run.

Ways to Optimize Older Content 

Some of your old content might not be optimized for SEO very well, rank for irrelevant keywords, or drive no traffic at all. If the quality is still decent, however, you should be able to optimize it properly with little effort. 

Advertisement

Refresh Content 

If your blog post contains a specific year or mentions current events, it may become outdated over time. If the rest of the content is still relevant (like if it’s targeting an evergreen topic), simply updating the date might be all you need to do.

Rewrite Old Blog Posts 

When the content quality is low (you might have greatly improved your writing skills since you’ve written the post) but the potential is still there, there’s not much you can do apart from rewriting an old blog post completely. 

This is not a waste—you’re saving time on brainstorming since the basic structure is already in place. Now, focus on improving the quality.

Delete Old Blog Posts 

You might find a blog post that just seems unusable. Should you delete your old content? It depends. If it’s completely outdated, of low quality, and irrelevant to any valuable keywords for your website, it’s better to remove it. 

Once you decide to delete the post, don’t forget to set up a 301 redirect to a related post or page, or to your homepage.

Promote Old Blog Posts 

Sometimes all your content needs is a bit of promotion to start ranking and getting traffic again. Share it on your social media, link to it from a new post – do something to get it discoverable again to your audience. This can give it the boost it needs to attract organic links too.

Advertisement

Which Blog Posts Should You Update?

Deciding when to update or rewrite blog posts is a decision that relies on one important thing: a content audit. 

Use your Google Analytics to find out which blog posts used to drive tons of traffic, but no longer have the same reach. You can also use Google Search Console to find out which of your blog posts have lost visibility in comparison to previous months. I have a guide on website analysis using Google Analytics and Google Search Console you can follow.

If you use keyword tracking tools like SE Ranking, you can also use the data it provides to come up with a list of blog posts that have dropped in the rankings. 

Make data-driven decisions to identify which blog posts would benefit from these updates – i.e., which ones still have the chance to recover their keyword rankings and organic traffic. 

With Google’s helpful content update, which emphasizes better user experiences, it’s crucial to ensure your content remains relevant, valuable, and up-to-date.

How To Update Old Blog Posts for SEO

Updating articles can be an involved process. Here are some tips and tactics to help you get it right.

Advertisement

Author’s Note: I have a Comprehensive On-Page SEO Checklist you might also be interested in following while you’re doing your content audit.

Conduct New Keyword Research

Updating your post without any guide won’t get you far. Always do your keyword research to understand how users are searching for your given topic. 

Proper research can also show you relevant questions and sections that can be added to the blog post you’re updating or rewriting. Make sure to take a look at the People Also Ask (PAA) section that shows up when you search for your target keyword. Check out other websites like Answer The Public, Reddit, and Quora to see what users are looking for too. 

Look for New Ranking Opportunities

When trying to revive an old blog post for SEO, keep an eye out for new SEO opportunities (e.g., AI Overview, featured snippets, and related search terms) that didn’t exist when you first wrote your blog post. Some of these features can be targeted by the new content you will add to your post, if you write with the aim to be eligible for it. 

Rewrite Headlines and Meta Tags

If you want to attract new readers, consider updating your headlines and meta tags. 

Your headlines and meta tags should fulfill these three things:

Advertisement
  1. Reflect the rewritten and new content you’ve added to the blog post.
  2. Be optimized for the new keywords it’s targeting (if any).
  3. Appeal to your target audience – who may have changed tastes from when the blog post was originally made. 

Remember that your meta tags in particular act like a brief advertisement for your blog post, since this is what the user first sees when your blog post is shown in the search results page. 

Take a look at your blog post’s click-through rate on Google Search Console – if it falls below 2%, it’s definitely time for new meta tags. 

Replace Outdated Information and Statistics

Updating blog content with current studies and statistics enhances the relevance and credibility of your post. By providing up-to-date information, you help your audience make better, well-informed decisions, while also showing that your content is trustworthy.

Tighten or Expand Ideas

Your old content might be too short to provide real value to users – or you might have rambled on and on in your post. It’s important to evaluate whether you need to make your content more concise, or if you need to elaborate more. 

Keep the following tips in mind as you refine your blog post’s ideas:

  • Evaluate Helpfulness: Measure how well your content addresses your readers’ pain points. Aim to follow the E-E-A-T model (Experience, Expertise, Authoritativeness, Trustworthiness).
  • Identify Missing Context: Consider whether your content needs more detail or clarification. View it from your audience’s perspective and ask if the information is complete, or if more information is needed.
  • Interview Experts: Speak with industry experts or thought leaders to get fresh insights. This will help support your writing, and provide unique points that enhance the value of your content.
  • Use Better Examples: Examples help simplify complex concepts. Add new examples or improve existing ones to strengthen your points.
  • Add New Sections if Needed: If your content lacks depth or misses a key point, add new sections to cover these areas more thoroughly.
  • Remove Fluff: Every sentence should contribute to the overall narrative. Eliminate unnecessary content to make your post more concise.
  • Revise Listicles: Update listicle items based on SEO recommendations and content quality. Add or remove headings to stay competitive with higher-ranking posts.

Improve Visuals and Other Media

No doubt that there are tons of old graphics and photos in your blog posts that can be improved with the tools we have today. Make sure all of the visuals used in your content are appealing and high quality. 

Update Internal and External Links

Are your internal and external links up to date? They need to be for your SEO and user experience. Outdated links can lead to broken pages or irrelevant content, frustrating readers and hurting your site’s performance.

You need to check for any broken links on your old blog posts, and update them ASAP. Updating your old blog posts can also lead to new opportunities to link internally to other blog posts and pages, which may not have been available when the post was originally published.

Optimize for Conversions

When updating content, the ultimate goal is often to increase conversions. However, your conversion goals may have changed over the years. 

Advertisement

So here’s what you need to check in your updated blog post. First, does the call-to-action (CTA) still link to the products or services you want to promote? If not, update it to direct readers to the current solution or offer.

Second, consider where you can use different conversion strategies. Don’t just add a CTA at the end of the post. 

Last, make sure that the blog post leverages product-led content. It’s going to help you mention your products and services in a way that feels natural, without being too pushy. Being subtle can be a high ROI tactic for updated posts.

Key Takeaway

Reviving old blog articles for SEO is a powerful strategy that can breathe new life into your content and boost your website’s visibility. Instead of solely focusing on creating new posts, taking the time to refresh existing content can yield impressive results, both in terms of traffic and conversions. 

By implementing these strategies, you can transform old blog posts into valuable resources that attract new readers and retain existing ones. So, roll up your sleeves, dive into your archives, and start updating your content today—your audience and search rankings will thank you!

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

How Compression Can Be Used To Detect Low Quality Pages

Published

on

By

Compression can be used by search engines to detect low-quality pages. Although not widely known, it's useful foundational knowledge for SEO.

The concept of Compressibility as a quality signal is not widely known, but SEOs should be aware of it. Search engines can use web page compressibility to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords, making it useful knowledge for SEO.

Although the following research paper demonstrates a successful use of on-page features for detecting spam, the deliberate lack of transparency by search engines makes it difficult to say with certainty if search engines are applying this or similar techniques.

What Is Compressibility?

In computing, compressibility refers to how much a file (data) can be reduced in size while retaining essential information, typically to maximize storage space or to allow more data to be transmitted over the Internet.

TL/DR Of Compression

Compression replaces repeated words and phrases with shorter references, reducing the file size by significant margins. Search engines typically compress indexed web pages to maximize storage space, reduce bandwidth, and improve retrieval speed, among other reasons.

This is a simplified explanation of how compression works:

  • Identify Patterns:
    A compression algorithm scans the text to find repeated words, patterns and phrases
  • Shorter Codes Take Up Less Space:
    The codes and symbols use less storage space then the original words and phrases, which results in a smaller file size.
  • Shorter References Use Less Bits:
    The “code” that essentially symbolizes the replaced words and phrases uses less data than the originals.

A bonus effect of using compression is that it can also be used to identify duplicate pages, doorway pages with similar content, and pages with repetitive keywords.

Research Paper About Detecting Spam

This research paper is significant because it was authored by distinguished computer scientists known for breakthroughs in AI, distributed computing, information retrieval, and other fields.

Advertisement

Marc Najork

One of the co-authors of the research paper is Marc Najork, a prominent research scientist who currently holds the title of Distinguished Research Scientist at Google DeepMind. He’s a co-author of the papers for TW-BERT, has contributed research for increasing the accuracy of using implicit user feedback like clicks, and worked on creating improved AI-based information retrieval (DSI++: Updating Transformer Memory with New Documents), among many other major breakthroughs in information retrieval.

Dennis Fetterly

Another of the co-authors is Dennis Fetterly, currently a software engineer at Google. He is listed as a co-inventor in a patent for a ranking algorithm that uses links, and is known for his research in distributed computing and information retrieval.

Those are just two of the distinguished researchers listed as co-authors of the 2006 Microsoft research paper about identifying spam through on-page content features. Among the several on-page content features the research paper analyzes is compressibility, which they discovered can be used as a classifier for indicating that a web page is spammy.

Detecting Spam Web Pages Through Content Analysis

Although the research paper was authored in 2006, its findings remain relevant to today.

Then, as now, people attempted to rank hundreds or thousands of location-based web pages that were essentially duplicate content aside from city, region, or state names. Then, as now, SEOs often created web pages for search engines by excessively repeating keywords within titles, meta descriptions, headings, internal anchor text, and within the content to improve rankings.

Section 4.6 of the research paper explains:

Advertisement

“Some search engines give higher weight to pages containing the query keywords several times. For example, for a given query term, a page that contains it ten times may be higher ranked than a page that contains it only once. To take advantage of such engines, some spam pages replicate their content several times in an attempt to rank higher.”

The research paper explains that search engines compress web pages and use the compressed version to reference the original web page. They note that excessive amounts of redundant words results in a higher level of compressibility. So they set about testing if there’s a correlation between a high level of compressibility and spam.

They write:

“Our approach in this section to locating redundant content within a page is to compress the page; to save space and disk time, search engines often compress web pages after indexing them, but before adding them to a page cache.

…We measure the redundancy of web pages by the compression ratio, the size of the uncompressed page divided by the size of the compressed page. We used GZIP …to compress pages, a fast and effective compression algorithm.”

High Compressibility Correlates To Spam

The results of the research showed that web pages with at least a compression ratio of 4.0 tended to be low quality web pages, spam. However, the highest rates of compressibility became less consistent because there were fewer data points, making it harder to interpret.

Figure 9: Prevalence of spam relative to compressibility of page.

The researchers concluded:

Advertisement

“70% of all sampled pages with a compression ratio of at least 4.0 were judged to be spam.”

But they also discovered that using the compression ratio by itself still resulted in false positives, where non-spam pages were incorrectly identified as spam:

“The compression ratio heuristic described in Section 4.6 fared best, correctly identifying 660 (27.9%) of the spam pages in our collection, while misidentifying 2, 068 (12.0%) of all judged pages.

Using all of the aforementioned features, the classification accuracy after the ten-fold cross validation process is encouraging:

95.4% of our judged pages were classified correctly, while 4.6% were classified incorrectly.

More specifically, for the spam class 1, 940 out of the 2, 364 pages, were classified correctly. For the non-spam class, 14, 440 out of the 14,804 pages were classified correctly. Consequently, 788 pages were classified incorrectly.”

The next section describes an interesting discovery about how to increase the accuracy of using on-page signals for identifying spam.

Insight Into Quality Rankings

The research paper examined multiple on-page signals, including compressibility. They discovered that each individual signal (classifier) was able to find some spam but that relying on any one signal on its own resulted in flagging non-spam pages for spam, which are commonly referred to as false positive.

Advertisement

The researchers made an important discovery that everyone interested in SEO should know, which is that using multiple classifiers increased the accuracy of detecting spam and decreased the likelihood of false positives. Just as important, the compressibility signal only identifies one kind of spam but not the full range of spam.

The takeaway is that compressibility is a good way to identify one kind of spam but there are other kinds of spam that aren’t caught with this one signal. Other kinds of spam were not caught with the compressibility signal.

This is the part that every SEO and publisher should be aware of:

“In the previous section, we presented a number of heuristics for assaying spam web pages. That is, we measured several characteristics of web pages, and found ranges of those characteristics which correlated with a page being spam. Nevertheless, when used individually, no technique uncovers most of the spam in our data set without flagging many non-spam pages as spam.

For example, considering the compression ratio heuristic described in Section 4.6, one of our most promising methods, the average probability of spam for ratios of 4.2 and higher is 72%. But only about 1.5% of all pages fall in this range. This number is far below the 13.8% of spam pages that we identified in our data set.”

So, even though compressibility was one of the better signals for identifying spam, it still was unable to uncover the full range of spam within the dataset the researchers used to test the signals.

Combining Multiple Signals

The above results indicated that individual signals of low quality are less accurate. So they tested using multiple signals. What they discovered was that combining multiple on-page signals for detecting spam resulted in a better accuracy rate with less pages misclassified as spam.

Advertisement

The researchers explained that they tested the use of multiple signals:

“One way of combining our heuristic methods is to view the spam detection problem as a classification problem. In this case, we want to create a classification model (or classifier) which, given a web page, will use the page’s features jointly in order to (correctly, we hope) classify it in one of two classes: spam and non-spam.”

These are their conclusions about using multiple signals:

“We have studied various aspects of content-based spam on the web using a real-world data set from the MSNSearch crawler. We have presented a number of heuristic methods for detecting content based spam. Some of our spam detection methods are more effective than others, however when used in isolation our methods may not identify all of the spam pages. For this reason, we combined our spam-detection methods to create a highly accurate C4.5 classifier. Our classifier can correctly identify 86.2% of all spam pages, while flagging very few legitimate pages as spam.”

Key Insight:

Misidentifying “very few legitimate pages as spam” was a significant breakthrough. The important insight that everyone involved with SEO should take away from this is that one signal by itself can result in false positives. Using multiple signals increases the accuracy.

What this means is that SEO tests of isolated ranking or quality signals will not yield reliable results that can be trusted for making strategy or business decisions.

Takeaways

We don’t know for certain if compressibility is used at the search engines but it’s an easy to use signal that combined with others could be used to catch simple kinds of spam like thousands of city name doorway pages with similar content. Yet even if the search engines don’t use this signal, it does show how easy it is to catch that kind of search engine manipulation and that it’s something search engines are well able to handle today.

Here are the key points of this article to keep in mind:

Advertisement
  • Doorway pages with duplicate content is easy to catch because they compress at a higher ratio than normal web pages.
  • Groups of web pages with a compression ratio above 4.0 were predominantly spam.
  • Negative quality signals used by themselves to catch spam can lead to false positives.
  • In this particular test, they discovered that on-page negative quality signals only catch specific types of spam.
  • When used alone, the compressibility signal only catches redundancy-type spam, fails to detect other forms of spam, and leads to false positives.
  • Combing quality signals improves spam detection accuracy and reduces false positives.
  • Search engines today have a higher accuracy of spam detection with the use of AI like Spam Brain.

Read the research paper, which is linked from the Google Scholar page of Marc Najork:

Detecting spam web pages through content analysis

Featured Image by Shutterstock/pathdoc

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

New Google Trends SEO Documentation

Published

on

By

Google publishes new documentation for how to use Google Trends for search marketing

Google Search Central published new documentation on Google Trends, explaining how to use it for search marketing. This guide serves as an easy to understand introduction for newcomers and a helpful refresher for experienced search marketers and publishers.

The new guide has six sections:

  1. About Google Trends
  2. Tutorial on monitoring trends
  3. How to do keyword research with the tool
  4. How to prioritize content with Trends data
  5. How to use Google Trends for competitor research
  6. How to use Google Trends for analyzing brand awareness and sentiment

The section about monitoring trends advises there are two kinds of rising trends, general and specific trends, which can be useful for developing content to publish on a site.

Using the Explore tool, you can leave the search box empty and view the current rising trends worldwide or use a drop down menu to focus on trends in a specific country. Users can further filter rising trends by time periods, categories and the type of search. The results show rising trends by topic and by keywords.

To search for specific trends users just need to enter the specific queries and then filter them by country, time, categories and type of search.

The section called Content Calendar describes how to use Google Trends to understand which content topics to prioritize.

Advertisement

Google explains:

“Google Trends can be helpful not only to get ideas on what to write, but also to prioritize when to publish it. To help you better prioritize which topics to focus on, try to find seasonal trends in the data. With that information, you can plan ahead to have high quality content available on your site a little before people are searching for it, so that when they do, your content is ready for them.”

Read the new Google Trends documentation:

Get started with Google Trends

Featured Image by Shutterstock/Luis Molinero

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending