SEO
At Least 66.5% of Links to Sites in the Last 9 Years Are Dead (Ahrefs Study on Link Rot)
The web is constantly changing, and pages get removed or redirected. This makes links to these pages go to a broken page or possibly a page that’s not like the original. This phenomenon is called link rot.
Since January 2013, 66.5% of the links pointing to the 2,062,173 websites we sampled have rotted. We found another 6.45% with temporary errors. We don’t know if they’re still there or not.
This is even more complicated when it comes to SEO. Another 1.55% have other issues that prevent the links from being counted for the purposes of ranking.
That means a total of 74.5% of the links in our study are considered lost, with at least 66.5% being rotted.
Often, the links that no longer work are important. Check out this example of a website that was referenced in a U.S. Supreme Court case. Someone bought the domain and used it to make a statement.
In a previous study of legal journals and citations from 2014, 70% of the links within the journals and 50% of the URLs from U.S. Supreme Court decisions did not contain the originally cited material.
Another study from 2012 found that 30% of social media links were dead within two years.
Most of the previous studies are fairly small and contain older parts of the web. I assume a lot more of the older web is already gone, if not most of it. For example, most sites stopped using extensions like .html on URLs many years ago in favor of clean URLs. Most sites have also moved from HTTP to HTTPs.
Considering the above, we decided to do the largest link rot study ever. And it’s one of the only ones that cover the more recent version of the web.
Let’s dig into the data.
About the data
Ahrefs has been crawling the web since 2010. But for the purpose of this study, we’re only looking at the data from January 2013.
You can use the Backlinks report in Ahrefs’ Site Explorer to check the data for your own site. For Ahrefs, 26.9 million out of 174.3 million links have been lost. Just compare the numbers with the “Lost” filter applied vs. the numbers with the “All” filter applied.
There are a few cases we tag as lost that we don’t count as link rot. I’ll cover that below.
As I mentioned in the intro, at least 66.5% of links to the sampled websites have rotted in the last nine years.
The web is complex and messy, and some things change faster than others. I wanted to see how many sites have link rot—and what percentage of their links experience link rot. This is the distribution for the percentage of link rot by domain across the dataset.
There are a lot of small sites that don’t have much link rot. If we take out the smallest sites and only look at those with more than 10 live links, you’ll see that larger sites seem to have quite a bit of link rot.
As I mentioned in the intro, the number of links we consider lost when it comes to SEO is even higher—percentage-wise, it’s 74.5%. I also wanted to see the distribution for these across the dataset.
There are a lot of small sites that don’t have many lost links. If we take out the smallest sites and only look at those with more than 10 live links, you’ll see that larger sites seem to have lost quite a lot of their links.
Links can be lost for many reasons. We classify lost links in different ways at Ahrefs. Here are the most common reasons that links are lost:
- Dropped (47.7%)
- Link removed (34.2%)
- Crawl error (6.45%)
- 301/302 (5.99%)
- Not found (4.11%)
- Not canonical (0.82%)
- Noindex (0.73%)
- Broken redirect (0%)
Let’s look at each of those and why they happen.
47.7% of links are from dropped pages
These pages are removed from our index for various reasons.
Pages may be dropped because they can’t be crawled or indexed. In some cases, a domain may not exist anymore.
34.2% of links are removed
In this case, the pages still exist; they just no longer link to you.
It could be that someone removed the link during a content refresh, replaced your link with a different one, or removed the link due to company policies. Another possibility is that a competitor decided to no longer link to you.
6.45% of lost links are from crawl errors
When we encounter an error while trying to crawl a page, it will be put into this bucket.
If the page is accessible when it’s crawled again and the link is still there, it will be counted as live. If the page continues to “error,” we may drop it from the index.
We chose to not count crawl errors in the total for link rot. It’s likely that a portion of these links no longer exists, but others still do.
5.99% of links are lost due to redirected pages
The page containing the link has been redirected somewhere else.
Pages change locations for all kinds of reasons. Commonly, this is the result of some kind of website migration.
4.11% of links are pages that are not found
In this case, the linking page has been deleted. The content, including the link, is missing.
Occasionally, these pages may become live again or be redirected; in such situations, they will be added back or placed in the redirect bucket.
0.82% of links are lost because the page they were on is no longer canonical
The canonical specified by the page has changed.
The linking page has a “rel=canonical” tag to some other location. It could be a change from HTTP to HTTPs or some kind of standardization involving trailing slashes or parameters. This is usually nothing to be worried about. The page is simply changing how it wants to be indexed. These links have just shifted locations, going from one page to another.
0.73% of links are lost because their pages are marked “noindex”
The linking page is marked “noindex,” so we don’t count the links from it.
We did not count pages marked as noindex in the numbers for link rot. The link technically exists, but the page it’s on won’t be found in search engines and won’t pass any value.
A small number of links are lost due to broken redirects
In this case, we saw multiple redirects in a chain before. Now one of those redirects is broken. The link is, thus, kind of disconnected from the target.
This happens if:
- The redirect chain is broken – If any of the pages in the redirect chain fails to respond, it gets reported as a lost link.
- The redirect no longer exists (or is changed) – Let’s say you had a link from Site A → Site B, but the link was first redirected through one or more other URLs (e.g., Site A → Site C → Site B). If the linking site swapped this link out so that it linked directly (rather than going through a redirect chain), it would be reported as a lost link. The same applies if the final URL of the redirect is changed to redirect elsewhere.
What can you do about link rot?
A lot of the links you obtain may be lost over time. One way you can possibly get some of them back is with link reclamation.
In many cases, your old URLs have links from other websites. If they’re not redirected to the current pages, then those links are lost and no longer count for your pages. It’s not too late to do these redirects, and you can quickly reclaim any lost value. Think of this as the fastest link building you will ever do.
Here’s how to find those opportunities:
I usually sort this by “Referring domains.”
You can even use link rot to your advantage. Broken link building is a tactic that involves finding resources in your niche that are no longer live, then reaching out to site owners and letting them know about a resource you have that can replace the broken link.
Want to know how to do this for your site? Our head of content, Joshua Hardwick, has you covered with a process-oriented guide to broken link building.
Another way to help with link rot is to fix broken links on your own website. These are easily identified in the Site Audit Links report. Just remove the links or update the reference to a relevant page that exists.
You may also want to fix broken links from your site that point to other sites. I have trouble arguing for this for SEO and, generally, will deem it as a website health and maintenance task that is of pretty low priority.
However, you can argue that clicking these links is bad for user experience. Accordingly, you can prioritize the links that are more often clicked.
The list of broken links to external pages can also be found in the Links report. If you see zero broken external links as I do, it’s probably because you didn’t enable “Check HTTP status of external links” in your Site Audit crawl settings.
Final thoughts
Some companies and technologies have tried to help with link rot. Many of these solutions don’t really solve the problem of broken links or a changing web. Instead, they rely on archiving what was on the web so it can still be seen. For example, the Internet Archive has a Chrome extension that will show archives of pages if they’re broken.
Similarly, the CDN Cloudflare has an Always Online option that will first look for its own archived copy of a page that’s offline. But if that doesn’t exist, it will pull the most recent version from the Internet Archive.
If you use Brave browser, a broken page will have a message that lets you check for an archived version at archive.org.
The Law Library of Congress implemented an external archiving solution for the problem of link and reference rot in its legal research reports.
As always, message me on Twitter if you have any questions.
SEO
HubSpot Rolls Out AI-Powered Marketing Tools
HubSpot announced a push into AI this week at its annual Inbound marketing conference, launching “Breeze.”
Breeze is an artificial intelligence layer integrated across the company’s marketing, sales, and customer service software.
According to HubSpot, the goal is to provide marketers with easier, faster, and more unified solutions as digital channels become oversaturated.
Karen Ng, VP of Product at HubSpot, tells Search Engine Journal in an interview:
“We’re trying to create really powerful tools for marketers to rise above the noise that’s happening now with a lot of this AI-generated content. We might help you generate titles or a blog content…but we do expect kind of a human there to be a co-assist in that.”
Breeze AI Covers Copilot, Workflow Agents, Data Enrichment
The Breeze layer includes three main components.
Breeze Copilot
An AI assistant that provides personalized recommendations and suggestions based on data in HubSpot’s CRM.
Ng explained:
“It’s a chat-based AI companion that assists with tasks everywhere – in HubSpot, the browser, and mobile.”
Breeze Agents
A set of four agents that can automate entire workflows like content generation, social media campaigns, prospecting, and customer support without human input.
Ng added the following context:
“Agents allow you to automate a lot of those workflows. But it’s still, you know, we might generate for you a content backlog. But taking a look at that content backlog, and knowing what you publish is still a really important key of it right now.”
Breeze Intelligence
Combines HubSpot customer data with third-party sources to build richer profiles.
Ng stated:
“It’s really important that we’re bringing together data that can be trusted. We know your AI is really only as good as the data that it’s actually trained on.”
Addressing AI Content Quality
While prioritizing AI-driven productivity, Ng acknowledged the need for human oversight of AI content:
“We really do need eyes on it still…We think of that content generation as still human-assisted.”
Marketing Hub Updates
Beyond Breeze, HubSpot is updating Marketing Hub with tools like:
- Content Remix to repurpose videos into clips, audio, blogs, and more.
- AI video creation via integration with HeyGen
- YouTube and Instagram Reels publishing
- Improved marketing analytics and attribution
The announcements signal HubSpot’s AI-driven vision for unifying customer data.
But as Ng tells us, “We definitely think a lot about the data sources…and then also understand your business.”
HubSpot’s updates are rolling out now, with some in public beta.
Featured Image: Poetra.RH/Shutterstock
SEO
Holistic Marketing Strategies That Drive Revenue [SaaS Case Study]
Brands are seeing success driving quality pipeline and revenue growth. It’s all about building an intentional customer journey, aligning sales + marketing, plus measuring ROI.
Check out this executive panel on-demand, as we show you how we do it.
With Ryann Hogan, senior demand generation manager at CallRail, and our very own Heather Campbell and Jessica Cromwell, we chatted about driving demand, lead gen, revenue, and proper attribution.
This B2B leadership forum provided insights you can use in your strategy tomorrow, like:
- The importance of the customer journey, and the keys to matching content to your ideal personas.
- How to align marketing and sales efforts to guide leads through an effective journey to conversion.
- Methods to measure ROI and determine if your strategies are delivering results.
While the case study is SaaS, these strategies are for any brand.
Watch on-demand and be part of the conversation.
Join Us For Our Next Webinar!
Navigating SERP Complexity: How to Leverage Search Intent for SEO
Join us live as we break down all of these complexities and reveal how to identify valuable opportunities in your space. We’ll show you how to tap into the searcher’s motivation behind each query (and how Google responds to it in kind).
SEO
What Marketers Need to Learn From Hunter S. Thompson
We’ve passed the high-water mark of content marketing—at least, content marketing in its current form.
After thirteen years in content marketing, I think it’s fair to say that most of the content on company blogs was created by people with zero firsthand experience of their subject matter. We have built a profession of armchair commentators, a class of marketers who exist almost entirely in a world of theory and abstraction.
I count myself among their number. I have hundreds of bylines about subfloor moisture management, information security, SaaS pricing models, agency resource management. I am an expert in none of these topics.
This has been the happy reality of content marketing for over a decade, a natural consequence of the incentives created by early Google Search. Historically, being a great content marketer required precisely no subject matter expertise. It was enough to read widely and write quickly.
Mountains of organic traffic have been built on the backs of armchair commentators like myself. Time spent doing deep, detailed research was, generally speaking, wasted, because 80% of the returns came from simply shuffling other people’s ideas around and slapping a few keyword-targeted H2s in the right places.
But this doesn’t work today.
For all of its flaws, generative AI is an excellent, truly world-class armchair commentator. If the job-to-be-done is reading a dozen articles and how-to’s and turning them into something semi-original and fairly coherent, AI really is the best tool for the job. Humans cannot out-copycat generative AI.
Put another way, the role of the content marketer as a curator has been rendered obsolete. So where do we go from here?
Hunter S. Thompson popularised the idea of gonzo journalism, “a style of journalism that is written without claims of objectivity, often including the reporter as part of the story using a first-person narrative.”
In other words, Hunter was the story.
When asked to cover the rising phenomenon of the Hell’s Angels, he became a Hell’s Angel. During his coverage of the ‘72 presidential campaign, he openly supported his preferred candidate, George McGovern, and actively disparaged Richard Nixon. His chronicle of the Kentucky Derby focused almost entirely on his own debauchery and chaos-making—a story that has outlasted any factual account of the race itself.
In the same vein, content marketers today need to become their stories.
It’s a content marketing truism that it’s unreasonable to expect writers to become experts. There’s a superficial level of truth to that claim—no content marketer can acquire a decade’s worth of experience in a few days or weeks—but there are great benefits awaiting any company willing to challenge that truism very, very seriously.
As Thompson proved, short, intense periods of firsthand experience can yield incredible insights and stories. So what would happen if you radically reduced your content output and dedicated half of your content team’s time to research and experimentation? If their job was doing things worth writing about, instead of just writing? If skin-in-the-game, no matter how small, was a prerequisite of the role?
We’re already seeing this shift.
Every week, I see more companies hiring marketers who are true, bonafide subject matter experts (I include the Ahrefs content team here—for the majority of our team, “writing” is a skill secondary to a decade of hands-on search and marketing experience). They are expensive, hard to find, and in the era of AI, worth every cent.
I see a growing expectation that marketers will document their experiences and experiments on social media, creating meta-content that often outperforms the “real” content. I see more companies willing to share subjective experiences and stories, and avoid competing solely on the sharing of objective, factual information. I see companies spending money to promote the personal brands of in-house creators, actively encouraging parasocial relationships as their corporate brand accounts lay dormant.
These are ideas that made no sense in the old model of content marketing, but they make much more sense today. This level of effort is fast becoming the only way to gain any kind of moat, creating material that doesn’t already exist on a dozen other company blogs.
In the era of information abundance, our need for information is relatively easy to sate; but we have a near-limitless hunger for entertainment, and personal interaction, and weird, pattern-interrupting experiences.
Gonzo content marketing can deliver.
-
SEARCHENGINES6 days ago
Daily Search Forum Recap: September 12, 2024
-
WORDPRESS5 days ago
How to Connect Your WordPress Site to the Fediverse – WordPress.com News
-
GOOGLE7 days ago
Google Warns About Misuse of Its Indexing API
-
SEARCHENGINES5 days ago
Daily Search Forum Recap: September 13, 2024
-
SEO6 days ago
OpenAI Claims New “o1” Model Can Reason Like A Human
-
SEO4 days ago
The Expert SEO Guide To URL Parameter Handling
-
SEO5 days ago
SEO Experts Gather for a Candid Chat About Search [Podcast]
-
SEO6 days ago
How to Build a Fandom by Talent-Scouting Great Content
You must be logged in to post a comment Login