Connect with us

SEO

12 Important Image SEO Tips You Need To Know

Published

on

12 Important Image SEO Tips You Need To Know

Think about the last time you uploaded an image to your website. Chances are you downloaded it from a stock photography site, uploaded it to the backend of your site, and then inserted it to the page.

This makes a shining example of image optimization, right? Not quite.

You’ve added a giant bowling ball weight to your site that’s slowing down the page speed. And, search engines can’t read your images without alt text.

Let’s change that.

Over 20% of all U.S. web searches happen on Google Images, according to 2018 data from Jumpshot.

SEO amateurs and pros alike know that optimizing images for your website is notoriously worth the time spent.

Dan Morgan at WebSpection got one of his photos to rank #1 in Google Images for “best person in Cardiff” in less than four days by optimizing his image.

And, Robbie Richards generated 150,732 visits by adding image alt tags, compressing images, and a few other SEO tricks.

Without proper image optimization, you’re wasting a valuable SEO asset.

It’s like the search engines are giving away Oreos and milk for free. But, you only take the Oreo. When in reality, the Oreo is way better dunked in milk.

Image optimization creates many advantages such as better user experience, faster page load times, and additional ranking opportunities. And, it’s becoming an increasingly more important role.

As Matt Southern pointed out, Gary Illyes’ statement on image search in a recent Reddit chat:

“We simply know that media search is way too ignored for what it’s capable doing for publishers so we’re throwing more engineers at it as well as more outreach.”

But which factors are most important to ensure your images are findable and don’t slow down your site?

Here are 12 important image optimization tips you need to know.

1. Choose The Right Format

Decoding all the various image formats can feel like your first time ordering at Taco Bell. But, before you can start adding images to your site, you want to make sure you’ve chosen the best file type.

While there are many image formats to choose from, the PNG and JPEG are the most common for the web.

  • PNG: Produces better quality images, but comes with a larger file size.
  • JPEG: You may lose image quality, but you can adjust the quality level to find a good balance.
  • WebP: Choose lossless or lossy compression using this, the only image format supported by both Chrome and Firefox.

For me, PNG is the unsung hero of image formatting. But, for my daily use, PNG is the way to go then convert those into WebP.

Just be careful if you’re using .jpg images inside an inline SVG format as Google’s systems can’t index these.

2. Compress Your Images

Yep, hell hath no fury like a bloated web page after uploading an image that’s not compressed.

Search engines will look at your web page like you might look at a big vat of Crisco: You can’t seriously be considering putting that on your website, right?

According to HTTP Archive, images makeup on average 21% of a total webpage’s weight.

That’s why I highly recommend compressing your images before uploading to your site. You can do this in Photoshop or you can use a tool like TinyPNG.

TingPNG also has a WordPress plugin you can use too.

However, I prefer WP Smush as my WordPress plugin. It reduces the image file size without removing the quality.

Whatever plugin you use, make sure to find one that compresses the images externally on their servers. It reduces the load on your own site.

Or, take it a step further and use an image CDN that detects the device and optimizes the image prior to delivery. Cloudinary and Imgix are two options to try out.

Increasingly.com improved website speed by 33%/2 seconds by compressing images.

I mean, there’s just something sexy about faster page speed when you compress your images.

If you’re unsure how your images are affecting your page speed, I recommend using Google’s PageSpeed Insights tool.

3. Create Unique Images

You want your photos to pop on your site. If you fill your website with stock imagery, you’ll look unoriginal – like thousands of other sites that don’t stand out.

Too many websites are cluttered with the same generic stock photos.

Think about a corporate website, a consulting firm, a business that prides itself on customer service. All these websites use virtually the same looking stock image of a businessman smiling.

I’m sure you’ve seen one that looks like this:

ImageCompression

While you may have your stock images perfectly optimized, it won’t have the same impact or potential SEO benefits as an original, high-quality image.

The more original pictures you have, the better experience for the user and the better your odds are of ranking on relevant searches.

Keep in mind that large images are more likely to be featured in Google Discover.

As Google recommends in its Advanced SEO resource,

“Large images need to be at least 1200 px wide and enabled by the max-image-preview:large setting, or by using AMP.”

Do not use your logo as the image.

4. Beware Of Copyright

Regardless of the image files you choose to use, make sure there’s no copyright conflict.

The Postal Service is paying $3.5 million in an image copyright lawsuit. And, Skechers got sued for $2.5 million.

If Getty, Shutterstock, DepositFiles, or some other stock photo provider owns an image you use, and you don’t have a license to use it, then you’re risking an expensive lawsuit.

Under the Digital Millennium Copyright Act (DMCA), you could be issued a notice if you have violated any copyright issues. If the owner of a piece of content sees their content on your website, they can issue a DMCA Takedown which you must comply with.

Google Images allows you to filter results based on those available for reuse, and Mindy Weinstein shares 41 different websites to find free images.

5. Customize Image File Names

When it comes to SEO, creating descriptive, keyword-rich file names is absolutely crucial.

Not customizing your image file name is like getting a burrito with nothing in it. It just plain sucks.

Image file names alert Google and other search engine crawlers as to the subject matter of the image.

CustomImageFileNames

Typically, file names will look like “IMG_722019” or something similar. That’s like ordering from a menu in a different language. It doesn’t help Google.

Change the file name from the default to help the search engines understand your image and improve your SEO value.

This involves a bit of work, depending on how extensive your media library is, but changing the default image name is always a good idea.

Let’s pretend you have an image of chocolate for example.

I could name it simply “chocolate” but if you sell chocolate on your website, potentially every image can be named “chocolate-1,” “chocolate-2,” and so on.

I named this image “dark-chocolate-coffee” to let users and search engines understand the image.

6. Write SEO-Friendly Alt Text

Alt tags are a text alternative to images when a browser can’t properly render them. Similar to the title, the alt attribute is used to describe the contents of an image file.

When the image won’t load, you’ll get an image box with the alt tag present in the top left corner. Make sure they fit with the image and make the picture relevant.

Paying attention to alt tags is also beneficial to the overall on-page SEO strategy. You want to make sure that all other optimization areas are in place, but if the image fails to load for any reason, users will see what the image is supposed to be.

Plus, adding appropriate alt tags to the images on your website can help your website achieve better rankings in the search engines by associating keywords with images. Even Google has remarked on the value of alt text in images.

It provides Google with useful information about the subject matter of the image. We use this information to help determine the best image to return for a user’s query.

SEOFriendlyAltText

Alt text is required under the American Disabilities Act for individuals who are unable to view images themselves. A descriptive alt text can alert users exactly what is in the photo. For example, say you have a picture of chocolate on your website.

The alt text could read:

<img src=”chocolate-1.jpg” alt=”chocolate”/>

However, a better alternative text that describes the image would read:

<img src=”chocolate-1.jpg” alt=”dark chocolate coffee flavored bar”/>

Alt text is viewable in the cached text version of the page, aiding in its benefit to both users and the search engines. For further SEO value, the alt text can act as the anchor text of an internal link when the image links to a different page on the site.

7. Think About The Image File Structure

Google updated its Image Guidelines. One of the major updates they revealed was that they use the file path and file name to rank images.

Repeat: The file path and file name is an actual ranking factor.

For example, if you’re an ecommerce brand with multiple products, instead of placing all your product images into a generic /media/ folder, I would recommend structuring your subfolders to more category related topics like /shorts/ or /denim/.

8. Optimize Your Page Title & Description

Google also revealed that it uses your page title and description as part of its image search algorithm.

The Google support page states:

Check your page title

All of your basic on-page SEO factors like meta data, header tags, copy on the page, structured data, etc. affects the way Google ranks your images.

It’s like putting all your toppings on your burrito. It tastes way better with guac. So, make sure to add the guac for improving image rankings.

9. Define Your Dimensions

If you’re using AMP or PWAs, you are required to define your image dimensions in the source code.

However, if you’re not using either, it’s still a best practice to define the width and height. It provides a better user experience.

Plus, it allows the browsers to size the image before the CSS is loaded. This stops the page from jumping when it loads.

Image dimension attributes are also important for preventing Cumulative Layout Shift (CLS) issues that can interfere with your Core Web Vitals optimization.

Making sure that you include width and height attributes for every image and video element is key.

This tells the browser how much space to allocate for the resource and prevents that annoying content-shifting that lowers your CLS score. Learn more here.

10. Make Your Images Mobile-Friendly

Oh, mobile SEO. At its worst, it can give you a high bounce rate and low conversions. But, at its best, it can give you more ranking power and better user engagement.

Problem is, how do you optimize your images for the mobile-first index?

Luckily, Google offers guidance on best practices for images.

In short, you want to create responsive images. This means the image will scale with the size of the site whether the user is using desktop or mobile. It adjusts to the size of the device.

Mozilla offers a comprehensive guide on using the srcset and sizes attributes to provide the browser additional source images, enabling the display of identical image content resized for the device.

It is important to format this with a different part of the attribute value on each line, as demonstrated in this example from their resource:

<img srcset=”elva-fairy-480w.jpg 480w,

elva-fairy-800w.jpg 800w”

sizes=”(max-width: 600px) 480px,

800px”

src=”elva-fairy-800w.jpg”

alt=”Elva dressed as a fairy”>

Learn more about using srcset for responsive images here.

11. Add Images To Your Sitemap

Whether you’re adding your images to your sitemap or creating a new sitemap for images, you want images somewhere in your sitemaps.

Having your images in a sitemap greatly increases the chances of search engines crawling and indexing your images. Thus, results in more site traffic.

If you’re using WordPress, Yoast and RankMath offer a sitemap solution in their plugin.

12. Add Structured Data

Structured data markup your content types to guide google and search engines to deliver better visual results. Essentially, Google could serve your images as a rich result if you add structured data.

For example, if you’re using schema markup on a product page and you label the image as a product, Google could pair this image with a price tag. Search engines skip the algorithm and use the information provided in the structured data to provide the right image.

Image Optimization Key Takeaways

So, before you begin uploading your image to your site, make sure to follow the image optimization rituals from above.

The most important thing is to make sure the image and alternative text are relevant to the page. Other key takeaways:

  • Choose the right file format. PNGs are my favorite for screenshots.
  • Reduce file size for faster page load speed.
  • Make sure your on-page SEO elements (meta data, structured data, etc.) pair with your image.
  • For crawlability, create an image sitemap or make sure your images are featured in your sitemap.

Optimizing images are no joke. With advancements in voice search technology, media is a growing importance and your entire site will benefit from taking the steps above.

Happy optimizing!


Featured Image: Paulo Bobita/Search Engine Journal

All screenshots are taken by author




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

Technical SEO Checklist for 2024: A Comprehensive Guide

Published

on

Technical SEO Checklist 2024 Comprehensive Strategies

Technical SEO Checklist 2024 Comprehensive Strategies

With Google getting a whopping total of six algorithmic updates and four core updates in 2023, you can bet the search landscape is more complicated (and competitive) to navigate nowadays.

To succeed in SEO this year, you will need to figure out what items to check and optimize to ensure your website stays visible. And if your goal is to not just make your website searchable, but have it rank at the top of search engine results, this technical SEO checklist for 2024 is essential.

Webmaster’s Note: This is part one of our three-part SEO checklist for 2024. I also have a longer guide on advanced technical SEO, which covers best practices and how to troubleshoot and solve common technical issues with your websites.

Technical SEO Essentials for 2024

Technical SEO refers to optimizations that are primarily focused on helping search engines access, crawl, interpret, and index your website without any issues. It lays the foundation for your site to be properly understood and served up by search engines to users.

1. Website Speed Optimization

A site’s loading speed is a significant ranking factor for search engines like Google, which prioritize user experience. Faster websites generally provide a more pleasant user experience, leading to increased engagement and improved conversion rates.

Server Optimization

Often, the reason why your website is loading slowly is because of the server it’s hosted on. It’s important to choose a high-quality server that ensures quick loading times from the get-go so you skip the headache that is server optimization.

Google recommends keeping your server response time under 200ms. To check your server’s response time, you need to know your website’s IP address. Once you have that, use your command prompt.

In the window that appears, type ping, followed by your website’s IP address. Press enter and the window should show how long it took your server to respond. 

If you find that your server goes above the recommended 200ms loading time, here’s what you need to check:

  1. Collect the data from your server and identify what is causing your response time to increase. 
  2. Based on what is causing the problem, you will need to implement server-side optimizations. This guide on how to reduce initial server response times can help you here.
  3. Measure your server response times after optimization to use as a benchmark. 
  4. Monitor any regressions after optimization.

If you work with a hosting service, then you should contact them when you need to improve server response times. A good hosting provider should have the right infrastructure, network connections, server hardware, and support services to accommodate these optimizations. They may also offer hosting options if your website needs more server resources to run smoothly.

Website Optimization

Aside from your server, there are a few other reasons that your website might be loading slowly. 

Here are some practices you can do:

  1. Compressing images to decrease file sizes without sacrificing quality
  2. Minimizing the code, eliminating unnecessary spaces, comments, and indentation.
  3. Using caching to store some data locally in a user’s browser to allow for quicker loading on subsequent visits.
  4. Implementing Content Delivery Networks (CDNs) to distribute the load, speeding up access for users situated far from the server.
  5. Lazy load your web pages to prioritize loading the objects or resources only your users need.

A common tool to evaluate your website speed is Google’s PageSpeed Insights or Google Lighthouse. Both tools can analyze the content of your website and then generate suggestions to improve its overall loading speed, all for free. There are also some third-party tools, like GTMetrix, that you could use as well.

Here’s an example of one of our website’s speeds before optimization. It’s one of the worst I’ve seen, and it was affecting our SEO.

slow site speed score from GTMetrixslow site speed score from GTMetrix

So we followed our technical SEO checklist. After working on the images, removing render-blocking page elements, and minifying code, the score greatly improved — and we saw near-immediate improvements in our page rankings. 

site speed optimization results from GTMetrixsite speed optimization results from GTMetrix

That said, playing around with your server settings, coding, and other parts of your website’s backend can mess it up if you don’t know what you’re doing. I suggest backing up all your files and your database before you start working on your website speed for that reason. 

2. Mobile-First Indexing

Mobile-first Indexing is a method used by Google that primarily uses the mobile version of the content for indexing and ranking. 

It’s no secret that Google places a priority on the mobile users’ experience, what with mobile-first indexing being used. Beyond that, optimizing your website for mobile just makes sense, given that a majority of people now use their phones to search online.

This change signifies that a fundamental shift in your approach to your website development and design is needed, and it should also be part of your technical SEO checklist.

  1. Ensuring the mobile version of your site contains the same high-quality, rich content as the desktop version.
  2. Make sure metadata is present on both versions of your site.
  3. Verify that structured data is present on both versions of your site.

Tools like Google’s mobile-friendly test can help you measure how effectively your mobile site is performing compared to your desktop versions, and to other websites as well.

3. Crawlability & Indexing Check

Always remember that crawlability and Indexing are the cornerstones of SEO. Crawlability refers to a search engine’s ability to access and crawl through a website’s content. Indexing is how search engines organize information after a crawl and before presenting results.

  1. Utilizing a well-structured robots.txt file to communicate with web crawlers about which of your pages should not be processed or scanned.
  2. Using XML sitemaps to guide search engines through your site’s content and ensure that all valuable content is found and indexed. There are several CMS plugins you can use to generate your sitemap.
  3. Ensuring that your website has a logical structure with a clear hierarchy, helps both users and bots navigate to your most important pages easily. 

Google Search Console is the tool you need to use to ensure your pages are crawled and indexed by Google. It also provides reports that identify any problems that prevent crawlers from indexing your pages. 

4. Structured Data Markup

Structured Data Markup is a coding language that communicates website information in a more organized and richer format to search engines. This plays a strategic role in the way search engines interpret and display your content, enabling enhanced search results through “rich snippets” such as stars for reviews, prices for products, or images for recipes.

Doing this allows search engines to understand and display extra information directly in the search results from it.

Key Takeaway

With all the algorithm changes made in 2023, websites need to stay adaptable and strategic to stay at the top of the search results page. Luckily for you, this technical SEO checklist for 2024 can help you do just that. Use this as a guide to site speed optimization, indexing, and ensuring the best experience for mobile and desktop users.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Why Google Seems To Favor Big Brands & Low-Quality Content

Published

on

By

Why Google Seems To Favor Big Brands & Low-Quality Content

Many people are convinced that Google shows a preference for big brands and ranking low quality content, something that many feel has become progressively worse. This may not be a matter of perception, something is going on, nearly everyone has an anecdote of poor quality search results. The possible reasons for it are actually quite surprising.

Google Has Shown Favoritism In The Past

This isn’t the first time that Google’s search engine results pages (SERPs) have shown a bias that favored big brand websites. During the early years of Google’s algorithm it was obvious that sites with a lot of PageRank ranked for virtually anything they wanted.

For example, I remember a web design company that built a lot of websites, creating a network of backlinks, raising their PageRank to a remarkable level normally seen only in big corporate sites like IBM. As a consequence they ranked for the two-word keyword phrase, Web Design and virtually every other variant like Web Design + [any state in the USA].

Everyone knew that websites with a PageRank of 10, the highest level shown on Google’s toolbar, practically had a free pass in the SERPs, resulting in big brand sites outranking more relevant webpages. It didn’t go unnoticed when Google eventually adjusted their algorithm to fix this issue.

The point of this anecdote is to point out an instance of where Google’s algorithm unintentionally created a bias that favored big brands.

Here are are other  algorithm biases that publishers exploited:

  • Top 10 posts
  • Longtail “how-to” articles
  • Misspellings
  • Free Widgets in footer that contained links (always free to universities!)

Big Brands And Low Quality Content

There are two things that have been a constant for all of Google’s history:

  • Low quality content
  • Big brands crowding out small independent publishers

Anyone that’s ever searched for a recipe knows that the more general the recipe the lower the quality of recipe that gets ranked. Search for something like cream of chicken soup and the main ingredient for nearly every recipe is two cans of chicken soup.

A search for Authentic Mexican Tacos results in recipes with these ingredients:

  • Soy sauce
  • Ground beef
  • “Cooked chicken”
  • Taco shells (from the store!)
  • Beer

Not all recipe SERPs are bad. But some of the more general recipes Google ranks are so basic that a hobo can cook them on a hotplate.

Robin Donovan (Instagram), a cookbook author and online recipe blogger observed:

“I think the problem with google search rankings for recipes these days (post HCU) are much bigger than them being too simple.

The biggest problem is that you get a bunch of Reddit threads or sites with untested user-generated recipes, or scraper sites that are stealing recipes from hardworking bloggers.

In other words, content that is anything but “helpful” if what you want is a tested and well written recipe that you can use to make something delicious.”

Explanations For Why Google’s SERPs Are Broken

It’s hard not to get away from the perception that Google’s rankings for a variety of topics always seem to default to big brand websites and low quality webpages.

Small sites grow to become big brands that dominate the SERPs, it happens. But that’s the thing, even when a small site gets big, it’s now another big brand dominating the SERPs.

Typical explanations for poor SERPs:

  • It’s a conspiracy to increase ad clicks
  • Content itself these days are low quality across the board
  • Google doesn’t have anything else to rank
  • It’s the fault of SEOs
  • Affiliates
  • Poor SERPs is Google’s scheme to drive more ad clicks
  • Google promotes big brands because [insert your conspiracy]

So what’s going on?

People Love Big Brands & Garbage Content

The recent Google anti-trust lawsuit exposed the importance of the Navboost algorithm signals as a major ranking factor. Navboost is an algorithm that interprets user engagement signals to understand what topics a webpage is relevant for, among other things.

The idea of using engagement signals as an indicator of what users expect to see makes sense. After all, Google is user-centric and who better to decide what’s best for users than the users themselves, right?

Well, consider that arguably the the biggest and most important song of 1991, Smells Like Teen Spirt by Nirvana, didn’t make the Billboard top 100 for that year. Michael Bolton and Rod Stewart made the list twice, with Rod Stewart top ranked for a song called “The Motown Song” (anyone remember that one?)

Nirvana didn’t make the charts until the next year…

My opinion, given that we know that user interactions are a strong ranking signal, is that Google’s search rankings follow a similar pattern related to users’ biases.

People tend to choose what they know. It’s called a Familiarity Bias.

Consumers have a habit of choosing things that are familiar over those that are unfamiliar. This preference shows up in product choices that prefer brands, for example.

Behavioral scientist, Jason Hreha, defines Familiarity Bias like this:

“The familiarity bias is a phenomenon in which people tend to prefer familiar options over unfamiliar ones, even when the unfamiliar options may be better. This bias is often explained in terms of cognitive ease, which is the feeling of fluency or ease that people experience when they are processing familiar information. When people encounter familiar options, they are more likely to experience cognitive ease, which can make those options seem more appealing.”

Except for certain queries (like those related to health), I don’t think Google makes an editorial decision to certain kinds of websites, like brands.

Google uses many signals for ranking. But Google is strongly user focused.

I believe it’s possible that strong user preferences can carry a more substantial weight than Reviews System signals. How else to explain why Google seemingly has a bias for big brand websites with fake reviews rank better than honest independent review sites?

It’s not like Google’s algorithms haven’t created poor search results in the past.

  • Google’s Panda algorithm was designed to get rid of a bias for cookie cutter content.
  • The Reviews System is a patch to fix Google’s bias for content that’s about reviews but aren’t necessarily reviews.

If Google has systems for catching low quality sites that their core algorithm would otherwise rank, why do big brands and poor quality content still rank?

I believe the answer is that is what users prefer to see those sites, as indicated by user interaction signals.

The big question to ask is whether Google will continue to rank what users biases and inexperience trigger user satisfaction signals.  Or will Google continue serving the sugar-frosted bon-bons that users crave?

Should Google make the choice to rank quality content at the risk that users find it too hard to understand?

Or should publishers give up and focus on creating for the lowest common denominator like the biggest popstars do?



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Google Announces Gemma: Laptop-Friendly Open Source AI

Published

on

By

Google Announces Gemma: Laptop-Friendly Open Source AI

Google released an open source large language model based on the technology used to create Gemini that is powerful yet lightweight, optimized to be used in environments with limited resources like on a laptop or cloud infrastructure.

Gemma can be used to create a chatbot, content generation tool and pretty much anything else that a language model can do. This is the tool that SEOs have been waiting for.

It is released in two versions, one with two billion parameters (2B) and another one with seven billion parameters (7B). The number of parameters indicates the model’s complexity and potential capability. Models with more parameters can achieve a better understanding of language and generate more sophisticated responses, but they also require more resources to train and run.

The purpose of releasing Gemma is to democratize access to state of the art Artificial Intelligence that is trained to be safe and responsible out of the box, with a toolkit to further optimize it for safety.

Gemma By DeepMind

The model is developed to be lightweight and efficient which makes it ideal for getting it into the hands of more end users.

Google’s official announcement noted the following key points:

  • “We’re releasing model weights in two sizes: Gemma 2B and Gemma 7B. Each size is released with pre-trained and instruction-tuned variants.
  • A new Responsible Generative AI Toolkit provides guidance and essential tools for creating safer AI applications with Gemma.
  • We’re providing toolchains for inference and supervised fine-tuning (SFT) across all major frameworks: JAX, PyTorch, and TensorFlow through native Keras 3.0.
  • Ready-to-use Colab and Kaggle notebooks, alongside integration with popular tools such as Hugging Face, MaxText, NVIDIA NeMo and TensorRT-LLM, make it easy to get started with Gemma.
  • Pre-trained and instruction-tuned Gemma models can run on your laptop, workstation, or Google Cloud with easy deployment on Vertex AI and Google Kubernetes Engine (GKE).
  • Optimization across multiple AI hardware platforms ensures industry-leading performance, including NVIDIA GPUs and Google Cloud TPUs.
  • Terms of use permit responsible commercial usage and distribution for all organizations, regardless of size.”

Analysis Of Gemma

According to an analysis by an Awni Hannun, a machine learning research scientist at Apple, Gemma is optimized to be highly efficient in a way that makes it suitable for use in low-resource environments.

Hannun observed that Gemma has a vocabulary of 250,000 (250k) tokens versus 32k for comparable models. The importance of that is that Gemma can recognize and process a wider variety of words, allowing it to handle tasks with complex language. His analysis suggests that this extensive vocabulary enhances the model’s versatility across different types of content. He also believes that it may help with math, code and other modalities.

It was also noted that the “embedding weights” are massive (750 million). The embedding weights are a reference to the parameters that help in mapping words to representations of their meanings and relationships.

An important feature he called out is that the embedding weights, which encode detailed information about word meanings and relationships, are used not just in processing input part but also in generating the model’s output. This sharing improves the efficiency of the model by allowing it to better leverage its understanding of language when producing text.

For end users, this means more accurate, relevant, and contextually appropriate responses (content) from the model, which improves its use in conetent generation as well as for chatbots and translations.

He tweeted:

“The vocab is massive compared to other open source models: 250K vs 32k for Mistral 7B

Maybe helps a lot with math / code / other modalities with a heavy tail of symbols.

Also the embedding weights are big (~750M params), so they get shared with the output head.”

In a follow-up tweet he also noted an optimization in training that translates into potentially more accurate and refined model responses, as it enables the model to learn and adapt more effectively during the training phase.

He tweeted:

“The RMS norm weight has a unit offset.

Instead of “x * weight” they do “x * (1 + weight)”.

I assume this is a training optimization. Usually the weight is initialized to 1 but likely they initialize close to 0. Similar to every other parameter.”

He followed up that there are more optimizations in data and training but that those two factors are what especially stood out.

Designed To Be Safe And Responsible

An important key feature is that it is designed from the ground up to be safe which makes it ideal for deploying for use. Training data was filtered to remove personal and sensitive information. Google also used reinforcement learning from human feedback (RLHF) to train the model for responsible behavior.

It was further debugged with manual re-teaming, automated testing and checked for capabilities for unwanted and dangerous activities.

Google also released a toolkit for helping end-users further improve safety:

“We’re also releasing a new Responsible Generative AI Toolkit together with Gemma to help developers and researchers prioritize building safe and responsible AI applications. The toolkit includes:

  • Safety classification: We provide a novel methodology for building robust safety classifiers with minimal examples.
  • Debugging: A model debugging tool helps you investigate Gemma’s behavior and address potential issues.
  • Guidance: You can access best practices for model builders based on Google’s experience in developing and deploying large language models.”

Read Google’s official announcement:

Gemma: Introducing new state-of-the-art open models

Featured Image by Shutterstock/Photo For Everything



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS