Connect with us

SEO

The 10 Best AI Writers & Content Generators Compared

Published

on

AI Content creation tools are becoming more widely available since the development of GPT-3 (and its release through Open.ai) has made AI much more accessible.

To see just how good AI writers are, we selected 10 of the best content generators and road-tested them by comparing their output on the same topic.

We ran our own mini Turing Test while testing the AI content generators.

We asked our audience if they could tell the difference between machine and human-generated content through social media polls.

Are machines taking over the content industry? Read on and find out!

How GPT-3 Is Shaking Up Content Creation

GPT-3, the language prediction model, was introduced in May 2020 and is widely available for public use through Open AI.

The quality of GPT-3 output was a huge leap forward from GPT-2 toward asking a machine to write intelligible cohesive content.

The downside to the development of GPT-3 and vastly improved content creation tools: How do you keep pace with the output of a machine?

Google has been looking at solving the potential issues of the predicted explosion of AI content to ensure their search results don’t become swamped with low-quality content.

They recently restated that AI content is against their guidelines and updated their webmaster guidelines documentation.

How can they distinguish between a machine and just badly written human content?

The progress and development of AI writers and content tools is only just getting started.

Private investment in AI more than doubled last year to $93.5 billion in 2021.

This means that more machine learning tools are being developed that will become more integrated into the tools we use as marketers and SEO professionals.

Meta recently announced a new research project into next-generation AI.

They aim to create an AI that processes data like humans and would be indistinguishable from a human.

When that might be available is not known.

Machine domination aside, in this article, we will review a selection of the current best content creation tools to see how they compare.

And consider how they can help us do a more efficient job.

Will AI take over the content creation industry? Let’s look at the results of the tools we tested.

AI Content Creation Tools Compared

We fed all the content generator tools we tested were fed the same subject matter to generate a similar length of output.

With a meta self-referential irony, we used the simple phrase, “AI content creation.”

We compared the use of the tool and the quality of the content output. You can see that output in each of the screenshots below.

As part of the comparison, we also ran all the content through Copyscape to check for plagiarism. Only one content generator was flagged with issues.

The difference in styles was most surprising about comparing the output from all the tools.

We half expected more similar results when running the same key phrase through an AI tool, but clearly, every generator has its own variables for how content is written. Much like a variety of human writers.

Screenshot from Writesonic, May 2022AI content creation via writesonic

1. Writesonic

  • Free version up to full unlimited price plans.
  • Paid version reasonably priced.
  • 24 languages.

Writesonic is built on GPT-3 and claims the machine is trained on the content that the brands using the tool produce.

The generator is based around facilitating marketing copy, blog articles, and product descriptions. The generator can also provide content ideas and outlines and has a full suite of templates for different types of content.

We found Writesonic very easy to use and it didn’t take much work to get a full article straight out of the box.

The free plan is an option to try the basic tool with the choice to upgrade to get access to full functionality.

AI content creation via fraseScreenshot from Frase, May 2022AI content creation via frase

2. Frase

  • Paid only.
  • Cheap plans, but the credits are expensive.

Frase is a content assistant and targeted to content marketers and SEO professionals for faster and better productivity.

The tool is structured around a framework of content brief, content writing, content optimization, and content analytics.

The tool excels for research and brief outlines and the talking points tool is useful for structuring an article. A content brief can be prepared in minutes.

For content writing, the generator doesn’t produce a full article straight out of the box, and it needs some work to get the results. But, the quality of content output is high.

Frase is a useful tool for content marketers that can help to reduce the amount of time spent on writing with a competent writer in charge.

Copy.ai content creationScreenshot from Copy.ai, May 2022Copy.ai content creation

3. Copy.ai

  • Free plan and paid unlimited.
  • Paid plans very cheap.
  • 25 languages.

Designed to be an antidote to writer’s block, Copy.ai is a cheap and easy-to-use content generator.

Copy.ai provides templates across various content types such as blogs, ads, sales, websites, and social media. The generator also provides translation into 25 languages.

An unusual addition to their range of tools includes a baby name generator, but we didn’t manage to get any usable results for a baby name.

Although easy to use, it’s a tool for anyone producing volume content up to an average level but not for high-end content production.

The free plan is an option to try before you buy.

AI content creation via AI WriterScreenshot from AI Writer, May 2022AI content creation via AI Writer

4. AI Writer

  • Free trial and paid plans.
  • Flagged in plagiarism checks.

AI Writer pitches itself as SEO-friendly, producing fresh and relevant copy that can save you 50% of your writing time.

From our test, AI Writer is an easy tool to use, and you get an article written within minutes.

However, it was the only tool we tested that our Copyscape plagiarism check flagged.

The articles from the writer were not the most fluid or cohesive, and felt much like an article spinner.

Out of all content generators tested, this tool didn’t feel like it was at the same standard of output as the others.

Hyperwrite toolScreenshot from Hyperwrite, May 2022Hyperwrite tool

5. Hyperwrite

Hyperwrite’s claim is to use the most advanced AI generator. It’s one of the most basic tools to use and generate content and the only tool that is fully free to use.

From our test, we found the output to be surprisingly cohesive and fluid.

A useful part of the tool is that it can rewrite a sentence or make paragraphs longer to quickly restructure and build out content where needed.

The text output from Hyperwrite was the one that most people chose as being human written in our Turing Test on social media.

It’s also a free tool that offers some of the better quality of all the tools compared and is the perfect tool to test the ability and constraints of AI content generation.

INK toolScreenshot from INK, May 2022INK tool

6. INK

  • Free version and full paid plans.
  • Chrome extension.

INK is another AI-powered tool targeted at content marketers and SEO experts as a content assistant for faster output and optimized content.

INK has 60 templates based around advertising, growth, website, and writing, including YouTube, pain agitation, catchy subjects, and listicles.

There’s a focus on SEO and getting content to rank with tools that support optimization and a tool scoring system to rate how well your article is optimized.

In the hands of a professional writer, INK can be a useful tool to support output, but it won’t do the job for you.

The tool took some work to get the final output, which was sometimes questionable.

INK generously offers up to 10 articles free in a month, which provides plenty of scope to try before you buy.

Rytr AI Content generationScreenshot from Rytr, May 2022Rytr AI Content generation

7. Rytr

  • Free plan to unlimited plan.
  • Cheap price plans.
  • 30 languages.

Rytr is a full AI content generation tool built on GPT-3, emphasizing generating content that converts.

The tool has over 30 templates for marketing copy, blogs, and product descriptions. It also incorporates AIDA and PAS formulas to get the best results for copywriting.

The tool was quick and easy to use and we had a reasonable quality article in five minutes.

Rytr has a free plan with access to all the tools with a limit of 5,000 characters a month. The paid plans start very cheap if you want to progress and take more advantage of the tool.

Smart Copy toolScreenshot from Smart Copy, May 2022Smart Copy tool

8. Snazzy (Now Smart Copy By Unbounce)

  • Free plan to paid unlimited version.

Snazzy is powered by GPT-3 and their own proprietary machine learning to create a tool focused on landing page generation.

Unbounce acquired Snazzy and rebranded as Smart Copy and is now structured to complement and support Unbounce for seamless landing page creation.

A full range of tools are available, such as outlines, ad copy, product descriptions, and social media copy. But, the tool is pitched toward generating sales-led persuasion copy.

The results generated in our test for ‘AI content generation’ were somewhat unpredictable and not intelligible enough for an article. However, we didn’t test specifically for landing page copy.

Snazzy/Smart Copy offers a free plan with up to five credits a day to try to see if it works for you.

Long Shot toolScreenshot from Long Shot, May 2022Long Shot tool

9. Long Shot

  • Free version up to unlimited version.
  • Eight Languages.

Long shot pitches itself as an AI-powered long-form content assistant to produce SEO-friendly content built on a combination of GPT-3 and custom AI models.

It includes over 30 tools for keyword research, rephrasing, and fact-checking, and you can write in eight languages.

From our test, we found Long Shot easy to use to produce reasonable quality content.

One interesting point: When comparing all the content output from the range of tools we tested, this was the only generator that included any brand names such as Google’s RankBrain and Buzzsumo.

This detail made the content output quite believable that it could be human-written.

Long Shot offers a free plan with up to 10 credits a day.

Jasper content toolScreenshot from Jasper, May 2022Jasper content tool

10. Jasper

  • Paid plans only.
  • 25 languages.

Jasper (previously Jarvis) claims it will help you write faster, beat writer’s block, and rank better with SEO-optimized content.

They also claim to have consulted with SEO professionals and direct marketing experts to perfect how the AI generator writes content.

Jasper has over 50 templates for producing content, including AIDA, PAS, blogs, social media, and marketing.

From our experience, Jasper is another writing support tool and not one that writes full articles without input. With guidance, the content generated from the tool is very good.

The tool is easy to use and the quality was good; however, we found the content generated was limited to short articles.

Jasper doesn’t offer any free plans, but it offers a free trial for five days.

Be aware you have to input your credit card and you will get charged if you forget to cancel.

Packages are not cheap, so you would have to max out the five-day trial to see if it was worth the investment.

The Results Of Our AI Content Turing Test

The test we ran was a simple short poll to gauge opinion and not a statistically significant result of large numbers. But, the results we found and the comments were surprising.

The 10 Best AI Writers & Content Generators ComparedThe 10 Best AI Writers & Content Generators Compared

The 10 Best AI Writers & Content Generators ComparedThe 10 Best AI Writers & Content Generators Compared

The 10 Best AI Writers & Content Generators ComparedThe 10 Best AI Writers & Content Generators Compared

The 10 Best AI Writers & Content Generators ComparedThe 10 Best AI Writers & Content Generators Compared

We provided three examples of 100 words of content, all based on “AI content creation” and created from some of the tools above. We wrote the fourth snippet.

We asked our audience, “Can you tell which one is human-generated?”

The short result is that no one could distinguish between AI-generated content and the human-written paragraph.

The reasons that people offered to justify those (incorrect) choices were quite interesting:

comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human
comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human
comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human

Out of all the comments, only a few guessed that number 4 was the human-generated copy:

comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human
comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human
comparing AI generated content vs. humanImage from Twitter, May 2022comparing AI generated content vs. human

See the full Twitter thread here.

From all responses across Twitter, Facebook, and LinkedIn, most people thought that number 1 was the human-written text.

The second text had the least amount of responses. Not surprising as this was not the best quality snippet of generated content.

The actual human-generated text was number 4 and came in third place in votes.

How You Can Use AI In Content Marketing

There’s a lot of experimentation happening with GPT-3 with plenty of fun tools being produced. But, look past these novelty applications to see where they can really have an impact.

Yes, you can machine generate your Twitter feed, but social media is about interaction and engagement.

Yes, you can write an article, but is it good enough to put your name to or represent your brand?

What needs to be considered with AI content generation is that a tool is only as good as the person operating it.

They are excellent for productivity and speeding up content production. But, you need someone who knows their subject and is a good writer behind the wheel to get results worthy of using.

A content marketer can take advantage of AI as an efficiency tool to make repetitive tasks easier and output faster.

In those terms, AI will become more and more seamlessly integrated into marketing.

Where AI Content Does Work

  • For product descriptions at scale.
  • For meta descriptions at scale.
  • Sports results broadcasting.
  • To support a writer’s productivity.

Where AI Content Doesn’t Work

  • Producing well-researched content.
  • Creating data-driven content.
  • Having innovative and fresh ideas.
  • Thought leadership.

Will AI Take Over Content Creation?

Although it’s now almost impossible to tell the difference between human and machine-generated content, the level of that content won’t win any journalist awards.

A tool cannot make up for a lack of knowledge or ability. It can only enhance it.

The machine generates content output from what is input; therefore, it only regurgitates; it isn’t creating new ideas.

It’s ideal for some tasks, but not for high-level well-researched content or thought leadership. And this is where good researchers and writers will become more valuable.

You can be assured that content creation will go into overdrive with AI.

You can also be assured that good quality journalist standard content with unique data, thought opinions, and insights will become the only way to get visibility and sustain an audience.

Who’s the winner of that game?

More resources: 


Featured Image: ProStockStudio/Shutterstock

!function(f,b,e,v,n,t,s)
{if(f.fbq)return;n=f.fbq=function(){n.callMethod?
n.callMethod.apply(n,arguments):n.queue.push(arguments)};
if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version=’2.0′;
n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];
s.parentNode.insertBefore(t,s)}(window,document,’script’,
‘https://connect.facebook.net/en_US/fbevents.js’);

if( typeof sopp !== “undefined” && sopp === ‘yes’ ){
fbq(‘dataProcessingOptions’, [‘LDU’], 1, 1000);
}else{
fbq(‘dataProcessingOptions’, []);
}

fbq(‘init’, ‘1321385257908563’);

fbq(‘track’, ‘PageView’);

fbq(‘trackSingle’, ‘1321385257908563’, ‘ViewContent’, {
content_name: ‘ai-writers-content-generators’,
content_category: ‘creation digital-marketing-tools ‘
});

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

SEO

Mozilla VPN Security Risks Discovered

Published

on

By

Mozilla VPN Security Risks Discovered

Mozilla published the results of a recent third-party security audit of its VPN services as part of it’s commitment to user privacy and security. The survey revealed security issues which were presented to Mozilla to be addressed with fixes to ensure user privacy and security.

Many search marketers use VPNs during the course of their business especially when using a Wi-Fi connection in order to protect sensitive data, so the  trustworthiness of a VNP is essential.

Mozilla VPN

A Virtual Private Network (VPN), is a service that hides (encrypts) a user’s Internet traffic so that no third party (like an ISP) can snoop and see what sites a user is visiting.

VPNs also add a layer of security from malicious activities such as session hijacking which can give an attacker full access to the websites a user is visiting.

There is a high expectation from users that the VPN will protect their privacy when they are browsing on the Internet.

Mozilla thus employs the services of a third party to conduct a security audit to make sure their VPN is thoroughly locked down.

Security Risks Discovered

The audit revealed vulnerabilities of medium or higher severity, ranging from Denial of Service (DoS). risks to keychain access leaks (related to encryption) and the lack of access controls.

Cure53, the third party security firm, discovered and addressed several risks. Among the issues were potential VPN leaks to the vulnerability of a rogue extension that disabled the VPN.

The scope of the audit encompassed the following products:

  • Mozilla VPN Qt6 App for macOS
  • Mozilla VPN Qt6 App for Linux
  • Mozilla VPN Qt6 App for Windows
  • Mozilla VPN Qt6 App for iOS
  • Mozilla VPN Qt6 App for Androi

These are the risks identified by the security audit:

  • FVP-03-003: DoS via serialized intent
  • FVP-03-008: Keychain access level leaks WG private key to iCloud
  • VP-03-010: VPN leak via captive portal detection
  • FVP-03-011: Lack of local TCP server access controls
  • FVP-03-012: Rogue extension can disable VPN using mozillavpnnp (High)

The rogue extension issue was rated as high severity. Each risk was subsequently addressed by Mozilla.

Mozilla presented the results of the security audit as part of their commitment to transparency and to maintain the trust and security of their users. Conducting a third party security audit is a best practice for a VPN provider that helps assure that the VPN is trustworthy and reliable.

Read Mozilla’s announcement:
Mozilla VPN Security Audit 2023

Featured Image by Shutterstock/Meilun

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Link Building Outreach for Noobs

Published

on

Link Building Outreach for Noobs

Link outreach is the process of contacting other websites to ask for a backlink to your website.

For example, here’s an outreach email we sent as part of a broken link building campaign:

In this guide, you’ll learn how to get started with link outreach and how to get better results. 

How to do link outreach

Link outreach is a four-step process:

1. Find prospects

No matter how amazing your email is, you won’t get responses if it’s not relevant to the person you’re contacting. This makes finding the right person to contact equally as important as crafting a great email.

Who to reach out to depends on your link building strategy. Here’s a table summarizing who you should find for the following link building tactics:

As a quick example, here’s how you would find sites likely to accept your guest posts:

  1. Go to Content Explorer
  2. Enter a related topic and change the dropdown to “In title”
  3. Filter for English results
  4. Filter for results with 500+ words
  5. Go to the “Websites” tab
Finding guest blogging opportunities via Content ExplorerFinding guest blogging opportunities via Content Explorer

This shows you the websites getting the most search traffic to content about your target topic.

From here, you’d want to look at the Authors column to prioritize sites with multiple authors, as this suggests that they may accept guest posts.

The Authors column indicate how many authors have written for the siteThe Authors column indicate how many authors have written for the site

If you want to learn how to find prospects for different link building tactics, I recommend reading the resource below.

2. Find their contact details

Once you’ve curated a list of people to reach out to, you’ll need to find their contact information.

Typically, this is their email address. The easiest way to find this is to use an email lookup tool like Hunter.io. All you need to do is enter the first name, last name, and domain of your target prospect. Hunter will find their email for you:

Finding Tim's email with Hunter.ioFinding Tim's email with Hunter.io

To prevent tearing your hair from searching for hundreds of emails one-by-one, most email lookup tools allow you to upload a CSV list of names and domains. Hunter also has a Google Sheets add-on to make this even easier.

Using the Hunter for Sheets add-on to find emails in bulk directly in Google SheetsUsing the Hunter for Sheets add-on to find emails in bulk directly in Google Sheets

3. Send a personalized pitch

Knowing who to reach out to is half the battle won. The next ‘battle’ to win is actually getting the person to care.

Think about it. For someone to link to you, the following things need to happen:

  • They must read your email
  • They must be convinced to check out your content
  • They must open the target page and complete all administrative tasks (log in to their CMS, find the link, etc.)
  • They must link to you or swap out links

That’s a lot of steps. Most people don’t care enough to do this. That’s why there’s more to link outreach than just writing the perfect email (I’ll cover this in the next section).

For now, let’s look at how to craft an amazing email. To do that, you need to answer three questions:

  1. Why should they open your email? — The subject line needs to capture attention in a busy inbox.
  2. Why should they read your email? — The body needs to be short and hook the reader in.
  3. Why should they link to you? — Your pitch needs to be compelling: What’s in it for them and why is your content link-worthy?

For example, here’s how we wrote our outreach email based on the three questions:

An analysis of our outreach email based on three questionsAn analysis of our outreach email based on three questions

Here’s another outreach email we wrote, this time for a campaign building links to our content marketing statistics post:

An analysis of our outreach email based on three questionsAn analysis of our outreach email based on three questions

4. Follow up, once

People are busy and their inboxes are crowded. They might have missed your email or read it and forgot.

Solve this by sending a short polite follow-up.

Example follow-up emailExample follow-up email

One is good enough. There’s no need to spam the other person with countless follow-up emails hoping for a different outcome. If they’re not interested, they’re not interested.

Link outreach tips

In theory, link outreach is simply finding the right person and asking them for a link. But there is more to it than that. I’ll explore some additional tips to help improve your outreach.

Don’t over-personalize

Some SEOs swear by the sniper approach to link outreach. That is: Each email is 100% customized to the person you are targeting.

But our experience taught us that over-personalization isn’t better. We ran link-building campaigns that sent hyper-personalized emails and got no results.

It makes logical sense: Most people just don’t do favors for strangers. I’m not saying it doesn’t happen—it does—but rarely will your amazing, hyper-personalized pitch change someone’s mind.

So, don’t spend all your time tweaking your email just to eke out minute gains.

Avoid common templates

My first reaction seeing this email is to delete it:

A bad outreach emailA bad outreach email

Why? Because it’s a template I’ve seen many times in my inbox. And so have many others.

Another reason: Not only did he reference a post I wrote six years ago, it was a guest post, i.e., I do not have control over the site. This shows why finding the right prospects is important. He even got my name wrong.

Templates do work, but bad ones don’t. You can’t expect to copy-paste one from a blog post and hope to achieve success.

A better approach is to use the scoped shotgun approach: use a template but with dynamic variables.

Email outreach template with dynamic variablesEmail outreach template with dynamic variables

You can do this with tools like Pitchbox and Buzzstream.

This can help achieve a decent level of personalization so your email isn’t spammy. But it doesn’t spend all your time writing customized emails for every prospect.

Send lots of emails

When we polled 800+ people on X and LinkedIn about their link outreach results, the average conversion rate was only 1-5%.

Link outreach conversion rates in 2023Link outreach conversion rates in 2023

This is why you need to send more emails. If you run the numbers, it just makes sense:

  • 100 outreach emails with a 1% success rate = 1 link
  • 1,000 outreach emails with a 1% success rate = 10 links

I’m not saying to spam everyone. But if you want more high-quality links, you need to reach out to more high-quality prospects.

Build a brand

A few years ago, we published a link building case study:

  • 515 outreach emails
  • 17.55% reply rate
  • 5.75% conversion rate

Pretty good results! Except the top comments were about how we only succeeded because of our brand:

Comments on our YouTube video saying we succeeded because of our brandComments on our YouTube video saying we succeeded because of our brand

It’s true; we acknowledge it. But I think the takeaway here isn’t that we should repeat the experiment with an unknown website. The takeaway is that more SEOs should be focused on building a brand.

We’re all humans—we rely on heuristics to make judgments. In this case, it’s branding. If your brand is recognizable, it solves the “stranger” problem—people know you, like you, and are more likely to link.

The question then: How do you build a brand?

I’d like to quote our Chief Marketing Officer Tim Soulo here:

What is a strong brand if not a consistent output of high-quality work that people enjoy? Ahrefs’ content team has been publishing top-notch content for quite a few years on our blog and YouTube channel. Slowly but surely, we were able to reach tens of millions of people and instill the idea that “Ahrefs’ content = quality content”—which now clearly works to our advantage.

Tim SouloTim Soulo

Ahrefs was once unknown, too. So, don’t be disheartened if no one is willing to link to you today. Rome wasn’t built in a day.

Trust the process and create incredible content. Show it to people. You’ll build your brand and reputation that way.

Build relationships with people in your industry

Outreach starts before you even ask for a link.

Think about it: People don’t do favors for strangers but they will for friends. If you want to build and maintain relationships in the industry, way before you start any link outreach campaigns.

Don’t just rely on emails either. Direct messages (DMs) on LinkedIn and X, phone calls—they all work. For example, Patrick Stox, our Product Advisor, used to have a list of contacts he regularly reached out to. He’d hop on calls and even send fruit baskets.

Create systems and automations

In its most fundamental form, link outreach is really about finding more people and sending more emails.

Doing this well is all about building systems and automations.

We have a few videos on how to build a team and a link-building system, so I recommend that you check them out.

Final thoughts

Good link outreach is indistinguishable from good business development.

In business development, your chances of success will increase if you:

  • Pitch the right partners
  • Have a strong brand
  • Have prior relationships with them
  • Pitch the right collaboration ideas

The same goes for link outreach. Follow the principles above and you will see more success for your link outreach campaigns.

Any questions or comments? Let me know on Twitter X.



Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

SEO

Research Shows Tree Of Thought Prompting Better Than Chain Of Thought

Published

on

By

Research Shows Tree Of Thought Prompting Better Than Chain Of Thought

Researchers discovered a way to defeat the safety guardrails in GPT4 and GPT4-Turbo, unlocking the ability to generate harmful and toxic content, essentially beating a large language model with another large language model.

The researchers discovered that the use of tree-of-thought (ToT)reasoning to repeat and refine a line of attack was useful for jailbreaking another large language model.

What they found is that the ToT approach was successful against GPT4, GPT4-Turbo, and PaLM-2, using a remarkably low number of queries to obtain a jailbreak, on average less than thirty queries.

Tree Of Thoughts Reasoning

A Google research paper from around May 2022 discovered Chain of Thought Prompting.

Chain of Thought (CoT) is a prompting strategy used on a generative AI to make it follow a sequence of steps in order to solve a problem and complete a task. The CoT method is often accompanied with examples to show the LLM how the steps work in a reasoning task.

So, rather than just ask a generative AI like Midjourney or ChatGPT to do a task, the chain of thought method instructs the AI how to follow a path of reasoning that’s composed of a series of steps.

Tree of Thoughts (ToT) reasoning, sometimes referred to as Tree of Thought (singular) is essentially a variation and improvement of CoT, but they’re two different things.

Tree of Thoughts reasoning is similar to CoT. The difference is that rather than training a generative AI to follow a single path of reasoning, ToT is built on a process that allows for multiple paths so that the AI can stop and self-assess then come up with alternate steps.

Tree of Thoughts reasoning was developed in May 2023 in a research paper titled Tree of Thoughts: Deliberate Problem Solving with Large Language Models (PDF)

The research paper describes Tree of Thought:

“…we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving.

ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices.

Our experiments show that ToT significantly enhances language models’ problem-solving abilities…”

Tree Of Attacks With Pruning (TAP)

This new method of jailbreaking large language models is called Tree of Attacks with Pruning, TAP. TAP uses two LLMs, one for attacking and the other for evaluating.

TAP is able to outperform other jailbreaking methods by significant margins, only requiring black-box access to the LLM.

A black box, in computing, is where one can see what goes into an algorithm and what comes out. But what happens in the middle is unknown, thus it’s said to be in a black box.

Tree of thoughts (TAP) reasoning is used against a targeted LLM like GPT-4 to repetitively try different prompting, assess the results, then if necessary change course if that attempt is not promising.

This is called a process of iteration and pruning. Each prompting attempt is analyzed for the probability of success. If the path of attack is judged to be a dead end, the LLM will “prune” that path of attack and begin another and better series of prompting attacks.

This is why it’s called a “tree” in that rather than using a linear process of reasoning which is the hallmark of chain of thought (CoT) prompting, tree of thought prompting is non-linear because the reasoning process branches off to other areas of reasoning, much like a human might do.

The attacker issues a series of prompts, the evaluator evaluates the responses to those prompts and then makes a decision as to what the next path of attack will be by making a call as to whether the current path of attack is irrelevant or not, plus it also evaluates the results to determine the likely success of prompts that have not yet been tried.

What’s remarkable about this approach is that this process reduces the number of prompts needed to jailbreak GPT-4. Additionally, a greater number of jailbreaking prompts are discovered with TAP than with any other jailbreaking method.

The researchers observe:

“In this work, we present Tree of Attacks with Pruning (TAP), an automated method for generating jailbreaks that only requires black-box access to the target LLM.

TAP utilizes an LLM to iteratively refine candidate (attack) prompts using tree-of-thoughts reasoning until one of the generated prompts jailbreaks the target.

Crucially, before sending prompts to the target, TAP assesses them and prunes the ones unlikely to result in jailbreaks.

Using tree-of-thought reasoning allows TAP to navigate a large search space of prompts and pruning reduces the total number of queries sent to the target.

In empirical evaluations, we observe that TAP generates prompts that jailbreak state-of-the-art LLMs (including GPT4 and GPT4-Turbo) for more than 80% of the prompts using only a small number of queries. This significantly improves upon the previous state-of-the-art black-box method for generating jailbreaks.”

Tree Of Thought (ToT) Outperforms Chain Of Thought (CoT) Reasoning

Another interesting conclusion reached in the research paper is that, for this particular task, ToT reasoning outperforms CoT reasoning, even when adding pruning to the CoT method, where off topic prompting is pruned and discarded.

ToT Underperforms With GPT 3.5 Turbo

The researchers discovered that ChatGPT 3.5 Turbo didn’t perform well with CoT, revealing the limitations of GPT 3.5 Turbo. Actually, GPT 3.5 performed exceedingly poorly, dropping from 84% success rate to only a 4.2% success rate.

This is their observation about why GPT 3.5 underperforms:

“We observe that the choice of the evaluator can affect the performance of TAP: changing the attacker from GPT4 to GPT3.5-Turbo reduces the success rate from 84% to 4.2%.

The reason for the reduction in success rate is that GPT3.5-Turbo incorrectly determines that the target model is jailbroken (for the provided goal) and, hence, preemptively stops the method.

As a consequence, the variant sends significantly fewer queries than the original method…”

What This Mean For You

While it’s amusing that the researchers use the ToT method to beat an LLM with another LLM, it also highlights the usefulness of ToT for generating surprising new directions in prompting in order to achieve higher levels of output.

  • TL/DR Takeaways:
  • Tree of Thought prompting outperformed Chain of Thought methods
  • GPT 3.5 worked significantly poorly in comparison to GPT 4 in ToT
  • Pruning is a useful part of a prompting strategy
  • Research showed that ToT is superior to CoT in an intensive reasoning task like jailbreaking an LLM

Read the original research paper:

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically (PDF)

Featured Image by Shutterstock/THE.STUDIO

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending