For SEO experts, our toolkit is crucial. It’s how we make sure we can quickly and effectively assess how well our websites are performing. Using the best tools can put you way ahead of other SEOs. One example (and one tool I’ve personally been using for years) is Screaming Frog. It’s a powerful, straightforward, and insightful website crawler tool that’s indispensable for finding technical issues on your website.

And the good news is that it keeps getting better. Screaming Frog just released its 20th major version of the software, which includes new features based on feedback from SEO professionals.

Here are the main updates:

1. Custom JavaScript Snippets
2. Mobile Usability
3. N-Grams Analysis
4. Aggregated Anchor Text
5. Carbon Footprint & Rating

Custom JavaScript Snippets

One of the standout features in this release is the ability to execute custom JavaScript snippets during a crawl. This functionality expands the horizons for data manipulation and API communication, offering unprecedented flexibility.

Use Cases:

• Data Extraction and Manipulation: Gather specific data points or modify the DOM to suit your needs.
• API Communication: Integrate with APIs like OpenAI’s ChatGPT from within the SEO Spider.

Setting Up Custom JS Snippets:

• Navigate to `Config > Custom > Custom JavaScript`.
• Click ‘Add’ to create a new snippet or ‘Add from Library’ to select from preset snippets.

• Ensure JavaScript rendering mode is set via `Config > Spider > Rendering`.

Crawl with ChatGPT:

• Leverage the `(ChatGPT) Template` snippet, add your OpenAI API key and tailor the prompt to your needs.
• Follow our tutorial on ‘How To Crawl With ChatGPT’ for more detailed guidance.

Sharing Your Snippets:

• Export/import snippet libraries as JSON files to share with colleagues.
• Remember to remove sensitive data such as API keys before sharing.

Introducing Custom JavaScript Snippets to Screaming Frog SEO Spider version 20.0 significantly enhances the tool’s flexibility and power. Whether you’re generating dynamic content, interacting with external APIs, or conducting complex page manipulations, these snippets open a world of possibilities. 

Mobile Usability

In today’s mobile-first world, ensuring a seamless mobile user experience is imperative. Version 20.0 introduces extensive mobile usability audits through Lighthouse integration. 

With an ever-increasing number of users accessing websites via mobile devices, ensuring a seamless mobile experience is crucial. Google’s mobile-first indexing highlights the importance of mobile usability, which directly impacts your site’s rankings and user experience.

 Mobile Usability Features:

• New Mobile Tab: This tab includes filters for regular mobile usability issues such as viewport settings, tap target sizes, content sizing, and more.
• Granular Issue Details: Detailed data on mobile usability issues can be explored in the ‘Lighthouse Details’ tab.
• Bulk Export Capability: Export comprehensive mobile usability reports via `Reports > Mobile`.

Setup:

• Connect to the PSI API through `Config > API Access > PSI` or run Lighthouse locally.

Example Use Cases:

• Identify pages where content does not fit within the viewport.
• Flag and correct small tap targets and illegible font sizes.

With these new features, Screaming Frog SEO Spider version 20.0 streamlines the process of auditing mobile usability, making it more efficient and comprehensive. By integrating with Google Lighthouse, both via the PSI API and local runs, the tool provides extensive insights into the mobile performance of your website. Addressing these issues not only enhances user experience but also improves your site’s SEO performance.

N-grams Analysis

N-grams analysis is a powerful new feature that allows users to analyze phrase frequency across web pages. This can greatly enhance on-page SEO efforts and internal linking strategies.

Setting Up N-grams:

• Activate HTML storage by enabling ‘Store HTML’ or ‘Store Rendered HTML’ under `Config > Spider > Extraction`.
• View the N-grams in the lower N-grams tab.

Example Use Cases:

• Improving Keyword Usage: Adjust content based on the frequency of targeted N-grams.
• Optimizing Internal Links: Use N-grams to identify unlinked keywords and create new internal links.

Internal Linking Opportunities:

The N-grams feature provides a nuanced method for discovering internal linking opportunities, which can significantly enhance your SEO strategy and site navigation.

The introduction of N-grams analysis in Screaming Frog SEO Spider version 20 provides a tool for deep content analysis and optimization. By understanding the frequency and distribution of phrases within your content, you can significantly improve your on-page SEO and internal linking strategies.

Aggregated Anchor Text

Effective anchor text management is essential for internal linking and overall SEO performance. The aggregated anchor text feature in version 20.0 provides clear insights into how anchor texts are used across your site.

Using Aggregated Anchor Text:

• Navigate to the ‘Inlinks’ or ‘Outlinks’ tab.
• Utilize the new ‘Anchors’ filters to see aggregated views of anchor text usage.

Practical Benefits:

• Anchor Text Diversity: Ensure a natural distribution of anchor texts to avoid over-optimization.
• Descriptive Linking: Replace generic texts like “click here” with keyword-rich alternatives.

The aggregated anchor text feature provides powerful insights into your internal link structure and optimization opportunities. This feature is essential if you are looking to enhance your site’s internal linking strategy for better keyword relevance, user experience, and search engine performance.

Aligning with digital sustainability trends, Screaming Frog SEO Spider version 20.0 includes features to measure and optimize your website’s carbon footprint.

Key Features:

• Automatic CO2 Calculation: The SEO Spider now calculates carbon emissions for each page using the CO2.js library.
• Carbon Rating: Each URL receives a rating based on its emissions, derived from the Sustainable Web Design Model.
• High Carbon Rating Identification: Pages with high emissions are flagged in the ‘Validation’ tab.

Practical Applications:

• Resource Optimization: Identify and optimize high-emission resources.
• Sustainable Practices: Implement changes such as compressing images, reducing script sizes, and using green hosting solutions.

The integration of carbon footprint calculations in Screaming Frog SEO Spider signifies a growing recognition of digital sustainability. As more businesses adopt these practices, we can collectively reduce the environmental impact of the web while driving performance and user satisfaction.

Other Updates

In addition to major features, version 20.0 includes numerous smaller updates and bug fixes that enhance functionality and user experience.

Rich Result Validation Enhancements:

• Split Google Rich Result validation errors from Schema.org.
• New filters and columns provide detailed insights into rich result triggers and errors.

Enhanced File Types and Filters:

• Internal and external filters include new file types such as Media, Fonts, and XML.

Website Archiving:

• A new option to archive entire websites during a crawl is available under `Config > Spider > Rendering > JS`.

Viewport and Screenshot Configuration:

• Customize viewport and screenshot sizes to fit different audit needs.

API Auto Connect:

• Automatically connect APIs on start, making the setup process more seamless.

Resource Over 15MB Filter:

• A new validation filter flags resources over 15MB, which is crucial for performance optimization.

Page Text Export:

• Export all visible page text through the new `Bulk Export > Web > All Page Text` option.

Lighthouse Details Tab:

• The ‘PageSpeed Details’ tab has been renamed ‘Lighthouse Details’ to reflect its expanded role.

HTML Content Type Configuration:

• An ‘Assume Pages are HTML’ option helps accurately classify pages without explicit content types.

Bug Fixes and Performance Improvements:

• Numerous small updates and fixes enhance stability and reliability. 

Screaming Frog SEO Spider version 20.0 is a comprehensive update packed with innovative features and enhancements that cater to the evolving needs of SEO professionals like us. From advanced data extraction capabilities with Custom JavaScript Snippets to environmental sustainability with Carbon Footprint and Rating, this release sets a new benchmark in SEO auditing tools.

Key Takeaway

Add this to your toolbox, or update to version 20 to explore the rich array of new features from Screaming Frog to optimize your website’s SEO, usability, and sustainability. It’s a no-fuss tool with tons of features that will help you stay ahead of your competitors, and ensure your websites perform optimally in terms of user experience and search engine visibility.

Google introduces Search Console feature for online stores to easily manage shipping and return policies.

• Google now allows online stores to manage shipping and return policies via Search Console.
• This simplifies providing vital information to customers.
• The feature can potentially boost sales for retailers.

Google updated their documentation to reflect that it added eight new languages to its translated results feature, broadening the reach of publishers to an increasingly global scale, with automatic  translations to a site visitor’s native language.

Google Translated Results

Translated Results is a Google Search feature that will automatically translate the title link and meta description into the local language of a user, making a website published in one language available to a searcher in another language. If the searcher clicks on the link of a translated result the web page itself will also be automatically translated.

According to Google’s documentation for this feature:

“Google doesn’t host any translated pages. Opening a page through a translated result is no different than opening the original search result through Google Translate or using Chrome in-browser translation. This means that JavaScript on the page is usually supported, as well as embedded images and other page features.”

This feature benefits publishers because it makes their website available to a larger audience.

Search Feature Available In More Languages

Google’s documentation for this feature was updated to reflect that it is now available in eight more languages.

Users who speak the following languages will now have automatic access to a broader range of websites.

List Of Added Languages

• Arabic
• Gujarati
• Korean
• Persian
• Thai
• Turkish
• Urdu
• Vietnamese

Why Did It Take So Long?

It seems odd that Google didn’t already translate results into so many major languages like Turkish, Arabic or Korean. So I asked international SEO expert Christopher Shin (LinkedIn profile) about why it might have taken so long for Google to do this in the Korean language.

Christopher shared:

Google was always facing difficulties in the South Korean market as a search engine, and that has to do mainly with Naver and Kakao, formerly known as Daum.

But the whole paradigm shift to Google began when more and more students that went abroad to where Google is the dominant search engine came back to South Korea. When more and more students, travelers abroad etc., returned to Korea, they started to realize the strengths and weaknesses of the local search portals and the information capabilities these local portals provided. Laterally, more and more businesses in South Korea like Samsung, Hyundai etc., started to also shift marketing and sales to global markets, so the importance of Google as a tool for companies was also becoming more important with the domestic population.

Naver is still the dominant search portal, but not to retrieve answers to specific queries, rather for the purpose of shopping, reviews etc.

So I believe that market prioritization may be a big part as to the delayed introduction of Translated Google Search Results. And in terms of numbers, Korea is smaller with only roughly 52M nationwide and continues to decline due to poor birth rates.

Another big factor as I see it, has to do with the complexity of the Korean language which would make it more challenging to build out a translation tool that only replicates a simple English version. We use the modern Korean Hangeul but also the country uses Hanja, which are words from the Chinese origin. I used to have my team use Google Translate until all of them complained that Naver’s Papago does a better job, but with the introduction of ChatGPT, the competitiveness offered by Google was slim.”

Takeaway

It’s not an understatement to say that 2024 has not been a good year for publishers, from the introduction of AI Overviews to the 2024 Core Algorithm Update, and missing image thumbnails on recipe blogger sites, there hasn’t been much good news coming out of Google. But this news is different because it creates the opportunity for publisher content to be shown in even more languages than ever.

Read the updated documentation here:

Translated results in Google Search

Featured Image by Shutterstock/baranq