Connect with us


Google’s Chain of Thought Prompting Can Boost Today’s Best Algorithms



Google's Chain of Thought Prompting Can Boost Today's Best Algorithms

Google announced a breakthrough research in Natural Language Processing called Chain of Thought Prompting that raises the state of the art of advanced technologies like PaLM and LaMDA to what the researchers call a remarkable level.

The fact that Chain of Thought Prompting can improve PaLM and LaMDA at these significant rates is a big deal.

LaMDA and PaLM

The research conducted experiments using two language models, Language Model for Dialogue Applications (LaMDA) and Pathways Language Model (PaLM).

LaMDA is a model focused on conversation, like a chatbot but also can be used for many other applications that require speaking, dialogue.

PaLM is a model that follows what Google calls the Pathways AI architecture where a language model is trained to learn how to solve problems.

Previously machine learning models were trained to solve one kind of problem and they’d be set loose essentially to do that one thing really well. But in order to do something else Google would have to train a new model.

The Pathways AI architecture is a way to create a model that can solve problems that it hasn’t necessarily seen before.

As quoted in the Google PaLM explainer:


“…we’d like to train one model that can not only handle many separate tasks, but also draw upon and combine its existing skills to learn new tasks faster and more effectively.”

What it Does

The research paper lists three important breakthroughs for Chain of Thought Reasoning:

  1. It allows language models to break down complex multi-step problems into a sequence of steps
  2. The chain of the thought process allows engineers to peek into the process and when things go wrong, this allows them to identify where it went wrong and fix it
  3. Can solve math word problems, can accomplish commonsense reasoning and according to the research paper can (in principle) solve any word-based problem that a human can.

Multi-step Reasoning Tasks

The research gives an example of a multi-step reasoning task that language models are tested on:

“Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?

A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 – 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9.”

PaLM is a state of the art language model that is part of the Pathways AI architecture. It is so advanced it can explain why a joke is funny.

Yet, as advanced as PaLM is, the researchers claim that the Chain of Thought Prompting significantly improves these models, and that’s what makes this new research so worthy of taking note of.
Google explains it like this:

“Chain of thought reasoning allows models to decompose complex problems into intermediate steps that are solved individually.

Moreover, the language-based nature of chain of thought makes it applicable to any task that a person could solve via language.”

The research paper then goes on to note that standard prompting doesn’t really improve when the scale of the model is increased.

However with this new approach scale has a significant and notable positive impact on how well the model performs.


Chain of Thought Prompting was tested on both LaMDA and PaLM, using two mathematical word problem datasets.


These datasets are used by researchers as a way to compare results on similar problems for different language models.

Below are images of graphs showing the results of using Chain of Thought Prompting on LaMDA.

Chain of Thought Prompting and LaMDA

The results of scaling LaMDA on the MultiArith dataset shows that it resulted modest improvement. But LaMDA scores significantly higher when scaled with Chain of Thought Prompting.

The results on the GSM8K dataset show a modest improvement.

It’s a different story with the PaLM language model.

Chain of Thought Prompting and PaLM

Chain of Thought Prompting and PaLM

As can be seen in the graph above the gains from scaling PaLM with Chain of Thought Prompting are huge, and they are huge for both datasets  (MultiArith and GSM8K).

The researchers call the results remarkable and a new state of the art:

“On the GSM8K dataset of math word problems, PaLM shows remarkable performance when scaled to 540B parameters.

…combining chain of thought prompting with the 540B parameter PaLM model leads to new state-of-the-art performance of 58%, surpassing the prior state of the art of 55% achieved by fine-tuning GPT-3 175B on a large training set and then ranking potential solutions via a specially trained verifier.


Moreover, follow-up work on self-consistency shows that the performance of chain of thought prompting can be improved further by taking the majority vote of a broad set of generated reasoning processes, which results in 74% accuracy on GSM8K.”


The conclusion of a research paper is one of the most important parts to check for understanding if the research advances the state of the art or is a dead-end or needs more research.

Google’s research paper conclusion section has a strongly positive note.

It notes:

“We have explored chain of thought prompting as a simple and broadly applicable method for enhancing reasoning in language models.

Through experiments on arithmetic, symbolic, and commonsense reasoning, we find that chain of thought processing is an emergent property of model scale that allows sufficiently large language models to perform reasoning tasks that otherwise have flat scaling curves.

Broadening the range of reasoning tasks that language models can perform will hopefully inspire further work on language-based approaches to reasoning.”

What that means is that Chain of Thought Prompting may have the potential to provide Google with the ability to significantly improve their various language models, which in turn can lead to significant improvements in the kinds of things Google can do.


Read the Google AI Article

Language Models Perform Reasoning via Chain of Thought


Download and Read the Research Paper

Chain of Thought Prompting Elicits Reasoning in Large Language Models (PDF)

Source link

See also  Identity and attribution can give your first-party data strategy a boost
Click to comment

Leave a Reply

Your email address will not be published.


4 Clues From Google That Tell Us Everything



4 Clues From Google That Tell Us Everything

When I was a kid, my favorite mysteries were the Sherlock Holmes novels by Sir Arthur Conan Doyle.

They all had their great quotes, but one of my favorites was from The Hound of the Baskervilles when Holmes tells Watson:

“The world is full of obvious things which nobody by any chance ever observes.”

I want to think that if Sir Doyle were alive today as a digital marketer in paid search, he might rephrase this quote to say something more like: “Google always leaves a trail of breadcrumbs, straight to the future of PPC, if you just take the time to look.”

I recently decided to look, and the results I found were eye-opening.

I came across more breadcrumbs than I could count, and many of them led to different places.

However, a core group revealed a clear picture of what is to come for the PPC industry.

Clue 1: New Google Ads Scripts Experience

Scripts for Google Ads have been around almost as long as the platform itself.


However, ask around, and you may be hard-pressed to find a person who has consistently used Scripts in their PPC campaigns or anyone who has ever used any of them.

Google wants that to change.

Version 2 of the Google Ads Scripts experience has officially launched, and it’s a huge step forward by Google to bring this feature to the forefront and support its use with a robust information and training portal.

What it tells us: With Google’s push toward automation, it is imperative to understand that going along for the ride is not an option.

It’s becoming a necessity.

When launching, optimizing, and maintaining campaign performance as you scale budgets, it’s becoming increasingly challenging to stay on top of everything without some help.

With this new offering, Google makes a clear statement for the future of PPC, both near and far.

There will be increased attention to automating your PPC campaign work, and Google Scripts is here for you.


Clue 2: Acquisition Of Looker

Looker is a Business Intelligence (BI) tool used to chart, graph, and display data so you can recognize and act upon problems and opportunities alike.

This app falls in the same category as Tableau and Power BI by Microsoft.

Three years ago, Google acquired Looker for $2.6 billion.

This acquisition completed the marketing channel UI to data presentation pipeline that Google desperately needed.

Google had already built out Big Query years prior, which allowed them to own the data warehouse portion of the data pipeline, but they were still missing the BI portion.

The acquisition of Looker enabled Google to offer a full suite of data tools, from beginning to end, to their users.

Users no longer needed to venture outside the Google ecosystem to obtain platforms and applications necessary to run a marketing service with end-to-end management.

What it tells us: Dealing with structured data and larger datasets that live outside of the marketing channel UI will be the norm for digital marketers.


As a PPC manager, you may not have to become a certified data and analytics expert, but you will have to be comfortable updating data sets, managing your campaign, and manipulating data inside your chosen BI application.

Clue 3: Broad Match & Responsive Ad Expansion

Is it just me, or does Google try to push the “Broad Match” bid strategy and the “Responsive” ad setup option every chance they get?

When adding keywords to a new campaign, you’ll get a stern disclaimer if you don’t designate your keywords as broad match.

Or how about the red text status warning when viewing campaign keywords?

You think something is wrong, but it’s just a “warning” that you could get more conversions if you choose “broad match” keywords for your ad set.

Then you have to deal with display campaigns!

When setting up a new display campaign, Google hides the standard display ad option and forces you to create a responsive display ad.

What it tells us: The ”suggestions” Google recommends (which always gives up more control to Google) have been going on for more than a decade.


And all I have to do is point to Expanded Text Ads to show you how this all ends.

Google will take more control over our campaigns to the point where Google will do nearly everything from campaign setup to ad copywriting and bid strategy selection.

Clue 4:  Google Glasses Announcement at I/O 2022

The long-awaited return of “Google Glasses” (officially named Proto 29) was announced at the annual Google I/O event with a slick video presentation.

While the video was relatively light on specifics, it certainly got people talking about the potential use cases, namely the ability of the glasses to translate foreign languages.

What it tells us: Things are changing, and they always will be.

If you were hoping to become an expert in all the ad software and marketing tactics and then coast on those skills for the rest of your career, you would be very disappointed.

Once “Google Glasses” are released and become widely adopted, we will need to learn and create campaigns for an entirely new ad platform.

Not only that, but if you think Google just released this video to brag about a niche product that will never catch on at scale, you have another thing coming.


This was the digital equivalent of Google planting a flag and saying, “This market share is ours, and it’s gonna be big!”

So, you have two options.

You can bury your head in the sand and hope on a shooting star that you will never have to use this groundbreaking technology for your PPC work.

Or, you can look at this as an opportunity, set a Google alert for any news related to Google Glasses, and then start learning whatever you can to become a leader in this new field.

Clue 5: “Automatically Created Assets” Beta Feature

Seamlessly nestled between the “Bidding” and “Start and End Dates” tabs in the campaign menu, you will see the biggest clue for the future of PPC.

Google states that the “Automatically Created Assets” feature:

“…will allow Google to help you generate headlines, descriptions, and other assets using your content from your landing page, domain, and ads. Google will provide you with automated tools to customize your assets based on relevance for your keywords. This may improve ad relevance and performance.”

What it tells us: If you read the statement closely, you will realize this one feature changes everything.

With just one feature, Google can, in theory, find relevant keywords to bid on for your business, create headlines and descriptions for search ads, and point the ads to a relevant landing page.


If you didn’t notice, those actions make up the lion’s share of what a PPC Manager creates daily and will dramatically alter what they fundamentally do as a marketing professional.

The Future Of PPC

So, what does this all mean, and how will this affect the daily job duties of PPC marketers?

Data Tracking & Analysis

If you haven’t already noticed in your day-to-day work, making sure data is tagged, tracked, sorted, and graphed is a big part of the job.

This will become a more significant part of your day as these elements become more complicated and clean data becomes king.

You may not need to become a full-fledged data scientist, but you will definitely need to learn how to aggregate data and manipulate it in the future.

Managing The Systems That Manage Campaigns

The days of directly “pulling the levers” of a PPC campaign are numbered.

We might be setting up and managing the systems and machines that “pull” the levers for us.

From writing JavaScript code that runs based on thousands of input data points to designing a special app on Google Glasses, the indirect management of campaigns seems likely.


Automating The Work Will Become The Work

There’s no doubt that automating more tasks we perform now will be vital to the future of PPC.

The new Google Script experience is all about automation, but you know it can be serious work to drive automation if you have ever written a script.

With the “Automatically Created Assets” feature, it seems strikingly clear that playing a larger role in setting up the main website to contain the optimal components for Google to use in an automated fashion will be essential.

It may not be the role you set out to play, but it may just be the role you need to play in the future of PPC.

The End (And The Beginning)

I may be right about all these predictions, some of them or absolutely none.

But if nothing else, and if history is any guide, the PPC manager’s role in 10 years will look different than the role we all play now.

Just keep your eyes open for all the clues that Google provides and you’ll remain ahead of the curve.

More resources:


Featured Image: New Africa/Shutterstock


if( typeof sopp !== “undefined” && sopp === ‘yes’ ){
fbq(‘dataProcessingOptions’, [‘LDU’], 1, 1000);
fbq(‘dataProcessingOptions’, []);

fbq(‘init’, ‘1321385257908563’);

fbq(‘track’, ‘PageView’);

fbq(‘trackSingle’, ‘1321385257908563’, ‘ViewContent’, {
content_name: ‘ppc-future-google-clues’,
content_category: ‘paid-media-strategy pay-per-click’

Source link

See also  Landing Page SEO Best Practices & Tips For Success
Continue Reading

Subscribe To our Newsletter
We promise not to spam you. Unsubscribe at any time.
Invalid email address