SEO
Google Considers Reducing Webpage Crawl Rate
Google may reduce the frequency of crawling webpages as it grows more conscious of the sustainability of crawling and indexing.
This topic is discussed by Google’s Search Relations team, which is made up of John Mueller, Martin Splitt, and Gary Illyes.
Together, in the latest episode of the Search Off the Record podcast, they discuss what to expect from Google in 2022 and beyond.
Among the topics they address is crawling and indexing, which SEO professionals and website owners say they’ve seen less of over the past year.
That’s going to be a key focus for Google this year as it aims to make crawling more sustainable by conserving computing resources.
Here’s what that will mean for your website and its performance in search results.
Sustainability Of Crawling & Indexing
Since Googlebot crawling and indexing happens virtually, it’s not something you may think has an impact on the environment.
Illyes brings this issue to attention when he says computing isn’t sustainable in general:
“… what I mean is that computing, in general, is not really sustainable. And if you think of Bitcoin, for example, Bitcoin mining has real impact on the environment that you can actually measure, especially if the electricity is coming from coal plants or other less sustainable plants.
We are carbon-free, since I don’t even know, 2007 or something, 2009, but it doesn’t mean that we can’t reduce even more our footprint on the environment. And crawling is one of those things that early on, we could chop off some low-hanging fruits.”
The low-hanging fruits, in this instance, refers to unnecessary web crawling. Such as crawling webpages that haven’t had any recent updates.
How Will Google Make Crawling More Sustainable?
Illyes goes on to explain that web crawling can be made more sustainable by cutting down on refresh crawls.
There are two types of Googlebot crawling: crawling to discover new content and crawling to refresh existing content.
Google is considering scaling back on crawling to refresh content.
Illyes continues:
“… one thing that we do, and we might not need to do that much, is refresh crawls. Which means that once we discovered a document, a URL, then we go, we crawl it, and then, eventually, we are going to go back and revisit that URL. That is a refresh crawl.
And then every single time we go back to that one URL, that will always be a refresh crawl. Now, how often do we need to go back to that URL?”
He goes on to give an example of certain websites that warrant a significant number of refresh crawls for some parts of the site but not others.
A website like Wall Street Journal is constantly updating its homepage with new content, so it deserves a lot of refresh crawls.
However, WSJ is not likely updating its About page as frequently, so Google doesn’t need to keep doing refresh crawls on those types of pages.
“So you don’t have to go back there that much. And often, we can’t estimate this well, and we definitely have room for improvement there on refresh crawls. Because sometimes it just seems wasteful that we are hitting the same URL over and over again.
Sometimes we are hitting 404 pages, for example, for no good reason or no apparent reason. And all these things are basically stuff that we could improve on and then reduce our footprint even more.”
If Google were to cut down on refresh crawls, which is not 100% confirmed, here’s the impact that could have on your website.
What Does A Reduction In Crawl Rate Mean For Your Website?
There’s a belief out there that a high crawl rate is a positive SEO signal, even if you’re not updating your content as often as Google is crawling it.
That’s a misconception, Illyes says, as content will not necessarily rank better because it gets crawled more.
Mueller:
“So I guess that’s kind of also a misconception that people have inthat they think if a page gets crawled more, it’ll get ranked more. Is that correct that that’s a misconception, or is that actually true?”
Illyes:
“It’s a misconception.”
Mueller:
“OK, so no need to try to force something to be re-crawled if it doesn’t actually change. It’s not going to rank better.”
Again, it’s not confirmed that Google will reduce refresh crawls, but it’s an idea the team is actively considering.
If Google follows through on this idea it won’t be a bad thing for your website. More crawling does not mean better rankings.
Moreover, the idea is to learn which pages need refresh crawls and which pages do not. That means the pages you change more often will continue to be refreshed and updated in search results.
For more details on how Google plans to pull this off, listen to the full discussion in the podcast below (starting at the 2:40 mark):
Featured Image: Alena Veasey/Shutterstock
SEO
Google Offers Solutions for Inaccurate Product Pricing In Search
In the latest edition of Google’s SEO office-hours Q&A video, Senior Search Analyst John Mueller addressed concerns about inaccurate prices in search results.
His advice may be helpful if you’re having similar problems with Google displaying the wrong prices for your products.
Ensuring Accurate Product Prices in Organic Search Results
One of the questions focused on ensuring accurate prices are displayed in organic text results for their products.
In his response, Mueller drew attention to Google Merchant Center feeds.
“I’d recommend using the Merchant Center feeds if you can,” Mueller advised.
He pointed out that Merchant Center offers ways to submit pricing data directly.
He recommends retailers look into this option, implying it’s a low-effort way to ensure accurate prices across search results.
Mueller continues:
“There are ways to submit pricing information in Merchant Center that don’t require a lot of work, so check that out. If you can’t find ways to resolve this, then please drop us a note in the help forums with the details needed to reproduce the issue.”
For those unfamiliar, Google Merchant Center is a tool that allows businesses to upload their product data to Google, making it available for Shopping ads, free listings, and other Google services.
This gives retailers more control over how their product information, including prices, appears across Google’s ecosystem.
Addressing Currency Discrepancies in Rich Results
Another question during the session concerned wrong currencies showing up in rich results.
This can impact how shoppers interact with search results, leading to confused customers and lost sales.
Mueller said this problem stems from Google’s systems potentially viewing pages as duplicates, especially when content is nearly identical across different regional site versions.
He explained,
“Often this is a side effect of Google systems seeing the page as being mostly duplicate. For example, if you have almost exactly the same content on pages for Germany and Switzerland, our systems might see the pages as duplicates, even if there’s a different price shown.”
To resolve this issue, Mueller suggests:
- Differentiate content: Pages for different regions or currencies should have sufficiently different content to avoid being flagged as duplicates.
- Use Merchant Center: As with the previous question, Mueller recommended using Merchant Center feeds for pricing information instead of relying solely on structured data.
Key Takeaways
This Q&A highlights a common challenge for online stores: Getting Google to show the correct prices in search results.
To help Google get it right, retailers should:
- Think about using Google Merchant Center to feed in more accurate prices.
- Ensure different country product page versions aren’t too similar to avoid duplicate content problems.
- Monitor how prices and currencies look in search results and rich snippets.
- Use Google’s help forums if there are problems you can’t fix.
Listen to the full Q&A session below:
Featured Image: Tada Images/Shutterstock
SEO
Meta’s New Ad Tools Promise More Precise Customer Targeting
Meta is rolling out ad platform upgrades for Facebook and Instagram.
The updates, coming in the next few months, focus on boosting performance and customization through AI-powered campaign optimization.
New Features For Precise Value Definition
Meta is rolling out a new “Conversion Value Rules” tool to give advertisers more flexibility.
This feature lets you adjust the value of different customer actions or groups to your business within a single campaign.
Let’s say you know some customers tend to spend way more over time. Now, you can tell the system to bid higher for those folks without setting up a separate campaign.
Incremental Attribution Model
Meta plans to introduce a new optional attribution setting later this year. This feature will focus on what it terms “incremental conversions.”
Instead of maximizing the total number of attributed conversions, this new model aims to optimize ad delivery for conversions likely to occur only because of ad exposure.
In other words, the model identifies and targets potential customers who wouldn’t have converted without seeing the advertisement.
Initial tests of this feature have yielded positive results. Advertisers participating in these trials have observed an average increase of over 20% in incremental conversions.
Enhanced Analytics Integration
Meta is launching direct connections with external analytics platforms, starting now and continuing through 2025. They’re kicking off with Google Analytics and Northbeam and plan to add Triple Whale and Adobe later.
These connections let businesses share combined campaign data from different channels with Meta’s ad system. The goal is to give advertisers a complete picture of how their campaigns perform across various platforms.
By getting this broader data set, Meta expects to fine-tune its AI models and help advertisers run more effective campaigns.
Cross-Publisher Journey Optimization
Meta is using what it’s learned from its early connections with analytics tools to update its ad system. These changes consider how customers interact with ads across different platforms before purchasing.
Early tests of this update have been positive. On average, third-party analytics tools show a 30% increase in conversions attributed to Meta ads. However, advertisers might see higher costs per thousand impressions (CPMs).
Right now, this update is being applied to campaigns that aim to increase the number or value of conversions under the sales objective. Meta plans to extend this to other campaign objectives soon.
Google Analytics Integration: What It Means
The Google Analytics connection is big news for industry folks, as it could offer the following benefits:
- Unified view of Meta ads and overall site performance
- Better multi-touch attribution
- Insights to refine SEO strategy based on paid social impact
- Smarter budget decisions between paid social and SEO
- Easier reporting
- Cross-channel optimization opportunities
This integration blurs the lines between paid social, organic social, and SEO, offering a more holistic view of digital marketing efforts.
Why This Matters
As privacy changes shake up digital advertising, Meta’s updates address the need for more accurate, valuable insights.
The move towards AI-driven features and cross-channel integration marks a new era in ad sophistication.
To make the most of these updates, review your Meta ad strategy and clearly define your customer journey and value metrics.
Stay tuned for the rollout, and be ready to test these new features as they become available.
Featured Image: Cristian Valderas/Shutterstock
SEO
50 Bootstrapped SaaS Companies Dominating SEO in 2024
We analyzed the organic traffic growth of 1,600 SaaS companies to discover the SEO strategies that work best in 2024.
In this article, we’re focusing on bootstrapped software companies—those who have never raised external funding. All SEO data featured here comes piping hot from the Ahrefs API.
This is a list of bootstrapped SaaS companies ordered by estimated monthly organic traffic growth from August 2023 to August 2024. Did you make the cut?
Rank | Company | Traffic 2023 | Traffic 2024 | Traffic Growth | Change | Estimated Revenue |
---|---|---|---|---|---|---|
1 | Smallpdf | 41,145,810 | 55,455,141 | 14,309,331 | 35% | $17.5M |
2 | BuiltWith | 526,880 | 3,149,986 | 2,623,106 | 498% | $31.0M |
3 | Ahrefs | 2,171,168 | 4,271,413 | 2,100,245 | 97% | $100.0M |
4 | Surfshark | 992,020 | 1,510,739 | 518,719 | 52% | $20.0M |
5 | IPQS | 1,194,124 | 1,660,482 | 466,358 | 39% | $32.0M |
6 | ExpressVPN | 1,620,815 | 1,927,577 | 306,762 | 19% | $14.9M |
7 | OpenWeather | 203,917 | 470,687 | 266,770 | 131% | $5.5M |
8 | Clockify | 600,071 | 859,005 | 258,934 | 43% | $5.5M |
9 | BambooHR | 652,833 | 808,660 | 155,827 | 24% | $237.8M |
10 | Surfer | 45,197 | 189,261 | 144,064 | 319% | $37.5M |
11 | Doofinder | 55,317 | 184,746 | 129,429 | 234% | $75.0M |
12 | Oxylabs.io | 92,671 | 207,640 | 114,969 | 124% | $375.0M |
13 | Todoist | 545,900 | 639,398 | 93,498 | 17% | $20.3M |
14 | Toggl | 295,570 | 388,062 | 92,492 | 31% | $14.7M |
15 | JetBrains | 2,712,320 | 2,796,818 | 84,498 | 3% | $252.1M |
16 | Zadarma | 130,940 | 214,538 | 83,598 | 64% | $175.0M |
17 | MXToolbox | 738,595 | 802,633 | 64,038 | 9% | $9.0M |
18 | TeamGantt | 120,783 | 176,899 | 56,116 | 46% | $5.0M |
19 | Mailtrap | 103,564 | 156,873 | 53,309 | 51% | $5.5M |
20 | GetResponse | 192,850 | 243,514 | 50,664 | 26% | $129.2M |
21 | DeviceAtlas | 59,267 | 109,341 | 50,074 | 84% | $19.0M |
22 | Elfsight | 65,359 | 104,938 | 39,579 | 61% | $8.0M |
23 | Vector Magic | 169,389 | 199,343 | 29,954 | 18% | $10.0M |
24 | Referral Factory | 943 | 27,591 | 26,648 | 2826% | $5.5M |
25 | Crowdin | 17,602 | 43,155 | 25,553 | 145% | $17.5M |
26 | Kaspr | 10,476 | 33,515 | 23,039 | 220% | $5.5M |
27 | Wisepops | 13,698 | 36,295 | 22,597 | 165% | $3.0M |
28 | SavyCal | 11,926 | 33,984 | 22,058 | 185% | $5.5M |
29 | Voucherify | 37,111 | 58,832 | 21,721 | 59% | $5.5M |
30 | Buzzsprout | 97,961 | 118,040 | 20,079 | 21% | $617K |
31 | Meeting Room 365 | 5,738 | 25,637 | 19,899 | 347% | $500K |
32 | vFairs | 18,175 | 37,812 | 19,637 | 108% | $75.0M |
33 | Helpjuice | 73,249 | 92,177 | 18,928 | 26% | $6.0M |
34 | Yardi | 64,177 | 81,896 | 17,719 | 28% | $5.5B |
35 | SpyFu | 62,410 | 79,361 | 16,951 | 27% | $2.0M |
36 | Wappalyzer | 61,561 | 78,303 | 16,742 | 27% | $500K |
37 | Filemail | 142,226 | 158,133 | 15,907 | 11% | $1.0M |
38 | Float.com | 29,088 | 44,737 | 15,649 | 54% | $5.5M |
39 | Free online HTML editor | 29,523 | 44,269 | 14,746 | 50% | – |
40 | BrightLocal 🇺🇦 | 52,853 | 66,876 | 14,023 | 27% | $10.3M |
41 | ShortPixel | 6,192 | 20,006 | 13,814 | 223% | $10.0M |
42 | GetMyInvoices | 10,880 | 23,767 | 12,887 | 118% | $5.5M |
43 | Mangools | 67,615 | 80,324 | 12,709 | 19% | $5.5M |
44 | La Growth Machine | 13,668 | 26,321 | 12,653 | 93% | $5.5M |
45 | Aha! | 135,156 | 147,570 | 12,414 | 9% | $100.0M |
46 | GTranslate Inc. | 7,739 | 20,125 | 12,386 | 160% | $5.3M |
47 | Slickplan | 13,270 | 24,857 | 11,587 | 87% | $1.0M |
48 | Uptime.com | 28,164 | 39,394 | 11,230 | 40% | $5.5M |
49 | Homerun | 23,049 | 33,883 | 10,834 | 47% | $38.4M |
50 | Glassnode | 13,321 | 24,111 | 10,790 | 81% | $5.5M |
Top 5 bootstrapped SaaS companies
For each of the top five companies, I ran a five-minute analysis using Ahrefs Site Explorer to understand the SEO strategies driving their growth.
In a nutshell: free tools, programmatic content, and content localization feature heavily.
Smallpdf offers free tools for editing and converting PDF files. They have an estimated annual revenue of $17,500,000.
Smallpdf | 2023 | 2024 | Absolute change | Percent change |
---|---|---|---|---|
Organic traffic | 41,145,810 | 55,455,141 | 14,309,331 | 34.78% |
Organic pages | 3,367 | 3,506 | 139 | 4.13% |
Organic keywords | 466,048 | 416,115 | -49,933 | -10.71% |
Keywords in top 3 | 54,933 | 69,359 | 14,426 | 26.26% |
Smallpdf’s traffic growth comes entirely from—you guessed it—free PDF tools. Conversion tools (converting from PDF to Word Doc, from JPG to PDF, and so on) drive the majority of the site’s traffic:
These free tools target extremely high volume keywords, like pdf to word with an estimated 3.9M searches in Indonesia alone. Here’s the actual page that ranks so well:
These tools have also been localized to different languages, with the Spanish-, Indonesian- and Portuguese-targeted subfolders generating a large portion of the website’s traffic growth:
BuiltWith is a tool for discovering the technologies companies use to use to build their websites. BuiltWith has an estimated annual revenue of $31,000,000.
BuiltWith | 2023 | 2024 | Absolute change | Percent change |
---|---|---|---|---|
Organic traffic | 526,880 | 3,149,986 | 2,623,106 | 497.86% |
Organic pages | 81,615 | 93,716 | 12,101 | 14.83% |
Organic keywords | 126,541 | 111,371 | -15,170 | -11.99% |
Keywords in top 3 | 4,538 | 4,307 | -231 | -5.09% |
The biggest single driver of BuiltWith’s growth is the addition of a Korean language subfolder, builtwith.com/ko, housing localized versions of 191 website pages:
Other language subfolders (particularly /de, /es, and /fr) also generate a decent amount of traffic:
Oh hey, that’s us! Ahrefs is a powerful all-in-one SEO toolset, and the source of all the SEO data in this article.
Ahrefs | 2023 | 2024 | Absolute change | Percent change |
---|---|---|---|---|
Organic traffic | 2,171,168 | 4,271,413 | 2,100,245 | 96.73% |
Organic pages | 4,522 | 18,982 | 14,460 | 319.77% |
Organic keywords | 225,855 | 231,761 | 5,906 | 2.61% |
Keywords in top 3 | 14,291 | 20,618 | 6,327 | 44.27% |
We don’t need conjecture here. Our free AI writing tools are responsible for a big part of our organic traffic growth:
Our paragraph rewriter (113k visits per month), paraphrasing tool (99k), and instagram caption generator (73k) are particularly popular:
Try them out for yourselves here: https://ahrefs.com/writing-tools.
We also launched a programmatic SEO campaign, using Ahrefs data to rank the world’s most popular websites:
The localized versions of these pages—particularly Swedish and French—account for the majority of the traffic:
Surfshark is a VPN tool with an estimated annual revenue of $20,000,000.
Surfshark | 2023 | 2024 | Absolute change | Percent change |
---|---|---|---|---|
Organic traffic | 992,020 | 1,510,739 | 518,719 | 52.29% |
Organic pages | 2,610 | 3,964 | 1,354 | 51.88% |
Organic keywords | 205,194 | 248,267 | 43,073 | 20.99% |
Keywords in top 3 | 5,359 | 9,390 | 4,031 | 75.22% |
One of Surfshark’s fastest-growing pages is a free IP address checking tool:
But interestingly, most of the company’s organic growth has come from branded keywords, like surfshark discount code, surfshark deals, and surfshark login:
This is a good indication that the company has been investing heavily in brand marketing, increasing branded search queries as a byproduct.
Looking at the Paid search reports in Ahrefs, we can see that the company launched a substantial advertising campaign in the last couple of months (we’ve logged 876 separate ads):
Big, expensive brand advertising campaigns are not common for bootstrapped companies.
Although the fundraising data suggests that Surfshark has never raised outside capital, their website shares that “In 2022, Surfshark and Nord Security merged under one holding company to form a cybersecurity powerhouse while still operating independently.”
This suggests that Surfshark may not be bootstrapped in the truest sense (but I’ve left it in because it showcases an interesting—and expensive—way to grow organic traffic).
IPQS is a fraud, bot detection, and IP validation tool with an estimated annual revenue of $32,000,000.
IPQS | 2023 | 2024 | Absolute change | Percent change |
---|---|---|---|---|
Organic traffic | 1,194,124 | 1,660,482 | 466,358 | 39.05% |
Organic pages | 22,378 | 31,350 | 8,972 | 40.09% |
Organic keywords | 72,181 | 75,390 | 3,209 | 4.45% |
Keywords in top 3 | 5,092 | 7,065 | 1,973 | 38.75% |
All of IPQS’ traffic growth (and in fact, 57% of their total estimated organic traffic) comes from a single page, a free tool for looking up phone numbers:
This page manages to rank in first place for keywords like phone number lookup (353K monthly searches in the US), phone lookup (48K), and check this phone (35K):
This, despite just 84 referring domains and a design that probably looked good circa 2005:
Final thoughts
We’ll share more of these analyses in the coming weeks. Want us to include your company in the next round? Just fill out this short Google Form.
-
SEARCHENGINES5 days ago
Google August Core Update, Big Ranking Bug, DOJ Google Breakup, AI Overview Links, Gemini Research & More
-
SEARCHENGINES6 days ago
Daily Search Forum Recap: August 16, 2024
-
SEO6 days ago
SERP Tracking With Free Tools: A Beginner’s Toolkit
-
SEO6 days ago
Live Updates (Unrelated to August Core Update)
-
SEO4 days ago
Google’s “Information Gain” Patent For Ranking Web Pages
-
WORDPRESS5 days ago
11 Best Side Hustles to Take Up In 2024 For Extra Income
-
NEWS2 days ago
OpenAI Introduces Fine-Tuning for GPT-4 and Enabling Customized AI Models
-
WORDPRESS6 days ago
Making Social Media Fun Again – WordPress.com News
You must be logged in to post a comment Login