SEARCHENGINES

How BERT Helps With Google Search

Published

2 years ago

February 18, 2022

Google has posted a new video on how BERT helps Google Search understand language. Google has been using BERT in search since 2018, we only knew about it in 2019. That being said, the short video basically says it is about Google understanding the little words better.

Here is the video:

Here is the transcript if you don’t want to listen:

If a pancake recipe told you to “mix the batter with the banana,” you probably wouldn’t think to use the banana as a mixing spoon. But what’s obvious to humans — things like context, tone, and intention — are actually very difficult for computers to pick up on. At its core, a Google Search is about understanding language. In order to return the right information, Google doesn’t just need to know the definition of the words… it needs to know what they all mean when strung together in a specific order. And that includes the smaller words like “for” and “to.” And when you think about how many different meanings a single word can have… you start to see how writing a computer program that takes all these nuances into account is pretty tough. See? Case in point. “Pretty” here doesn’t mean beautiful, it means “very.” More and more, people talk to Google the way they think and speak. And, more and more — Google is getting better at understanding what they mean. One of the biggest leaps forward in the history of Search came about with the introduction of “Bidirectional Encoder Representations from Transformers” or as we like to call it, BERT. BERT is a machine-learning, model architecture that helps Google process language and understand the context in which it appears. Search used to process a query by pulling out the words it thought were most important. For example, if you said, “can you get medicine for someone pharmacy” you would have gotten general results about pharmacies and prescriptions because it would have essentially ignored the word “for.” But with BERT, the LITTLE words are taken into account and it changes things. Search now understands you want to know if you can pick up medicine… prescribed to someone else. But how do you train a language model to pick up context? There’s a big difference between knowing words and understanding meaning. The model learns context by applying the same fill-in-the-blank principles it takes to complete a Mad Libs. So we take a phrase. We hide about 20% of the input words. And then we make the computer guess the words that are missing. Over time, the model begins to understand different words have different meanings depending on what’s around them. And the order in which they appear in that text, really matters. So when you search something complex like, “Fly fishing bait to use for trout in september montana” Search knows all the little words are important and because it now takes them all into account, Google can tell you the perfect bait for that time of year. BERT isn’t foolproof, but since implementing it in 2019, it’s improved a lot of searches. We should always be able to learn about whatever we’re curious about. And that’s why Search will always be working to understand exactly what you’re truly asking.

Good video from Google about BERT:
“The little words are taken into account, and that changes things.”
And… “There’s a big difference between knowing words and understanding meaning.” https://t.co/boqZaUlYht
— Glenn Gabe (@glenngabe) February 17, 2022

I am surprised Google did not release this video when we wrote about how Google uses AI in search.

Forum discussion at Twitter.

SEARCHENGINES

The Industry Mourns The Loss Of Mark Irvine

Published

24 hours ago

May 5, 2024

Entireweb News Bot

I am deeply sad to report that Mark Irvine passed away unexpectedly last night. Mark was deeply involved in the search marketing community, known as being a brilliant paid search specialist who always spent his time helping others over his long career in the space.

Mark has been in the search marketing industry for well over a decade, seven years at WordStream and the last four or so years at SearchLab. He was always a bright and welcoming smile at industry events, sharing advice with colleagues and friends.

Navah Hopkins worked with Mark for many years and she sent me the following message:

“Do less”
These were the words Mark would always say to me when we worked together. Yet he was incapable of doing less. He brought so much love and care to everything he did.
I was lucky enough to work with Mark directly for about 5 years, and counted him a friend since then. Beyond being a brilliant data scientist and a natural leader, Mark was one of the most human humans I’ve ever known. He understood people at a fundamental level and always made sure others around him could shine.
Advertisement

Mark would often say that I was his dedicated PR team and even as I’m writing this detailing his accomplishments and his amazing (and all together too short) life, I can hear him laughing that I’m doing it again. Well – tough…just enjoy the love we have for you and sip those heavenly pumpkin spice lattes.

Mark was always put data first behind every major project at WordStream, Navah told me. He was a worldly person who traveled abroad to speak at many events. He was really good at connecting with people abroad. Mark was named the #1 PPC influencer in 2019 and was consistently was on that list year-after-year. PPCHero has recognized him as one of the top PPC Influencers every year since 2015, as did Microsoft and in 2019.

Here is Mark’s bio at SearchLabs.

He leaves behind his partner, Bobby Main and his mother, Virginia Hall.

Here are some early reactions to the sad news of Mark passing:

I can’t believe he’s gone….at least we know he has an eternity of pumpkin spice lattes and Big Brother: Heaven edition to look forward to.
Thank you for the grace and love you brought to all lucky enough to know you @MarkIrvine89. Rest in peace ♥️ pic.twitter.com/lpufASDPaO
Advertisement

— Navah Hopkins (@navahf) May 3, 2024

Y’all… I can’t even.
It’s impossible to think that we’re going to live in a world without @MarkIrvine89 in it. Infinitely thankful I got to spend the last 4 years working with him and deepening our friendship…
— Greg Gifford (@GregGifford) May 3, 2024

Like many others, just enjoyed time spent with Mark @Pubcon earlier this year.
A brilliant paid search marketer, and a terrific human being, with red wine wit, and caviar personality.
Love you and miss you my friend. https://t.co/ntQzrS7jIg
— Damon Gochneaur (@DamonGochneaur) May 3, 2024
Advertisement

Just saw Mark at PubCon. Cant believe he’s gone. He was a PPC legend and a friend. 💔 #ppcchat https://t.co/nF0cbQ9U51
— Kirk Williams (@PPCKirk) May 3, 2024

I can’t believe Mark is gone – feels like it was just a few weeks ago that we were all hanging out at PubCon. I can’t believe he’s gone. Mark was always kind & generous in sharing his time and talents with the PPC community, and was a genuinely wonderful person.
Thoughts to… https://t.co/G2IcsbVtCy
— Sam (@DigitalSamIAm) May 3, 2024

I’m still in nothing but shock. You were an amazing person and friend. I’m going to miss you so much. Rest in peace @MarkIrvine89 🥺💔
— Chrissabur (@ChristoferAlec) May 3, 2024

Oh so heavy hearted. Your intellect, wit, and candor were next level and you never ceased to make others smile, laugh, and deeply ponder. You were a pillar of the PPC community. May you rest in peace, @MarkIrvine89 https://t.co/ir2fOqLPOb pic.twitter.com/RRFImGlXfa
— Akvile DeFazio (@AkvileDeFazio) May 3, 2024

I’m so very sorry to see this 😔 RIP @MarkIrvine89 🙏 https://t.co/4tn8R5LbMR
— Aleyda Solis 🕊️ (@aleyda) May 3, 2024
Advertisement

This is devastating news.
Mark was a truly lovely human and the world is a little less bright without him in it.
Massive hugs to his family and all who knew and loved him. He will be missed. 💔#PPCChat https://t.co/wa3u7EKRo4
— Julie F Bacchini (@NeptuneMoon) May 3, 2024

Really sad to hear the news about Mark. I remember sharing a number of posts from Mark over the years. I never got to meet him in person, but I knew he was a super-sharp paid search professional. Big loss for the industry. https://t.co/orXg1n19fY
— Glenn Gabe (@glenngabe) May 3, 2024

😢😢😢 Heartbreaking. Thoughts are with his family & friends 🙏
— Greg Finn (@gregfinn) May 3, 2024

I am gutted hearing this news. To think we were all together just 2 short months ago. He will always hold a special place in the PPC community, and for everyone he’s interacted with.
You had such a special relationship with Mark, no doubt. Sending you the biggest hug from MN.…
Advertisement

— Brooke Osmundson (@BrookeOsmundson) May 3, 2024

I’m very sorry to hear this news as well. Mark was a really nice person and a brilliant marketer. My condolences to his family and friends. https://t.co/nRxIekXBGy
— Lily Ray 😏 (@lilyraynyc) May 3, 2024

I am very much so at a loss of words. I met Mark while we spoke together at PubCon. I could feel it, he had a very kind soul. He will be missed. https://t.co/zj1gsdgGFn
— Ben Fisher (@TheSocialDude) May 3, 2024

What a truly wonderful human he was. This is just devastating and so heartbreaking. Sending you a huge hug and support, Navah, and my thoughts are with his family and loved ones.
— Ginny Marvin (@GinnyMarvin) May 3, 2024

Forum discussion at X.

SEARCHENGINES

Daily Search Forum Recap: May 3, 2024

Published

2 days ago

May 4, 2024

Entireweb News Bot

Here is a recap of what happened in the search forums today, through the eyes of the Search Engine Roundtable and other search forums on the web.

Google got their cost down to generate AI answers in search by 80%. Google is interested in alternatives to hreflang. Google may recrawl URLs multiple times per day or per month. Google is testing custom search filter templates in search. Google AdSense removed its privacy policy as a placement for withdrawal of consent. And I posted the weekly SEO video recap. And deeply sad to report that passing of Mark Irvine.

Search Engine Roundtable Stories:

Search News Buzz Video Recap: Google March Core Update Done, HCU Recoveries, Site Reputation Abuse & AI Topics

This week, we covered how the Google March 2024 core update finished back on April 19th, and Google forgot to tell us. I also posted the May 2024 Google webmaster report. Google said it’s still possible for sites to recover…

Google SGE AI Answers Now Cost 80% Less To Generate

One of the big worries for Google investors was the cost of running AI to show within its search results. The AI overviews, now AI answers are costly to run. But as expected, Google was able to reduce the cost to generate these AI Answers.

Google Search Tests Custom Filters & Templates

Google Search is testing custom filters and pre-define search filter templates. It appears you can save your custom filters and reuse those filters for future searches. This is showing up for a limited number of searchers.

Google Open To Alternative Ideas For Hreflang

Gary Illyes from Google said he has heard from the SEO industry that hreflang can be “annoying” and complex and confusing. So he is open to ideas on how to replace it and make it work for both small and really large websites.

Google Recrawls URLs At Different Rates: Multiple Times Per Day To Every Few Months

Google has to crawl the web to build its index for it to provide search results. And since webpages and documents get updated, Google needs to recrawl those URLs from time to time. Google will recrawl URLs at different rates, said John Mueller of Google.

Google AdSense Has Removed Privacy Policy As Place To Withdraw Consent

Google AdSense has removed reference to your privacy policy as a place to withdraw consent. AdSense now has clarified that the revocation link at the bottom of the page should say “Privacy and cookie settings”.

The Industry Mourns The Loss Of Mark Irvine – Brilliant & Giving Search Marketer

I am deeply sad to report that Mark Irvine passed away unexpectedly last night. Mark was deeply involved in the search marketing community, known as being a brilliant paid search specialist who always spent his time helping others over his long career in the space.

Google Gradient Canopy Signage

I honestly did not know the new Google office, or one of them at least, was named the Gradient Canopy. I mean, I guess it makes sense. Here is a sign in front of that building with that name.

Other Great Search Threads:

I like to think of these (artificial, impossible, philosophical) questions as “if I added X to a page, would it be better for Y?” Would adding SD to a page make it rank higher? No. Might it rank differently? Maybe – SD enables some search features, but th, John Mueller on X

Small Google doc change, they restructured the Product structured data documentation, Barry Schwartz on X

AdSense Earnings and Observations – May 2024, WebmasterWorld

I hope it’s not just technically well-made :-). There have been some interesting takes in the space, I’m sure someone will find a model that works for clients, users, search & is profitable., John Mueller on X

Once you launch a new campaign (and you already have a Conversion Tracking in place) Google will always re-direct you towards the Google Tag Setup with instructions how to implement it. Have you seen it too?, Thomas Eccel on X

We’ve added Salesforce as a data source for data import! You can now import data directly from Salesforce to Google Analytics as offline-event data. Schedule daily or hourly imports, and select any existing Salesforce mil, Google Analytics on X

Search Engine Land Stories:

Feedback:

Have feedback on this daily recap; let me know on Twitter @rustybrick or @seroundtable, on Threads, Mastodon and Bluesky and you can follow us on Facebook and on Google News and make sure to subscribe to the YouTube channel, Apple Podcasts, Spotify, Google Podcasts or just contact us the old fashion way.

SEARCHENGINES

Google Interested In Hreflang Alternatives

Published

3 days ago

May 3, 2024

Entireweb News Bot

Gary Illyes from Google said he has heard from the SEO industry that hreflang can be “annoying” and complex and confusing. So he is open to ideas on how to replace it and make it work for both small and really large websites.

He wrote this on LinkedIn saying, “Things I’ve learned and heard in Sofia at the SERPConf event.” He said one of those is that “hreflang is annoying.”

Gary wrote, “I don’t disagree,” that I guess he understands why SEOs and creators find it annoying.

So he said that he open to new ideas. He wrote, “I’m still very open to coming up with something less annoying, but it needs to work for small sites and mammoths as well, while delivering at least the same amount of information.”

So if you have ideas, let Gary know, he wrote, “Ping if you have ideas.”

You can use hreflang to tell Google about the variations of your content. This helps Google understand the various pages and how they are localized variations of the same content. But the implementation can be confusing and detailed, why he said it can be annoying. Several years ago, John Mueller said hreflang can be the most complex aspect in SEO.

Forum discussion at LinkedIn.