SEO

Claude Instant With 100k Tokens Outperforms Leading AI Chatbots

Published

1 year ago

June 4, 2023

Max

Claude Instant With 100k Tokens Outperforms Leading AI Chatbots

Anthropic released a new version of Claude that accepts 100,000 tokens, or approximately 75,000 words of input.

This allows users to analyze and perform tasks on lengthy excerpts of books, code, documents, transcripts, and more.

Screenshot from Poe, June 2023

Continue reading to learn more about the latest version of Anthropic AI chatbot, Claude-instant-100k, and see how it performs tasks compared to its competitors: ChatGPT, Bing AI, and Google Bard.

AI Chatbot Input Limitations

Want to ask questions about a piece of content? Here is the length of input you can submit to generative AI chatbots by OpenAI, Microsoft, and Google.

OpenAI’s standard GPT-4 model offers 8,000 context tokens, whereas the extended context-length model offers 32,000 tokens / 25,000 words of input.
The input box for the prompt on Bing AI chat shows 0/4000, suggesting it only accepts up to 4,000 characters. However, the chatbot accepted 40,400 characters of lorem ipsum and accurately counted that it was a total of 6,000 words.
Google Bard accepts up to 10,000 characters.

Claude-instant-100k accepts – three times more input than the best GPT-4 model.

But how well can they follow directions?

I experimented with three tasks using Claude-instant-100k, ChatGPT Plus, Bing AI, and Google Bard. Here are the results.

Task #1: Edit An 1,580-Word List With Claude, ChatGPT, Bing, And Bard

The input: A list of 126 items (some of the first ChatGPT plugins) with 1,850 words / 12,500 characters.

The prompt: Can you rephrase and edit the descriptions for the following list: [the list]

Claude Completed The Task

Claude finished editing my list at item #126 with no hallucinations and followed the directions to rephrase the item descriptions.
Compared to the other AI chatbots, Claude-instant-100k finished the task the fastest.

ChatGPT Default GPT-4 Hallucinated (Link To Chat)

ChatGPT stops midway through item #37.
I clicked Continue generating – ChatGPT finished #37 and stopped again.
I prompted it to Please continue.
ChatGPT begins listing items that were not in my original input.
After a few more clicks to Continue generating, ChatGPT finished editing my list at item #142.

Bing AI Tried

Bing accepted the entire prompt and list.
It asked if this was what I was looking for – I told it yes and to please continue.
After additional prompting it to please continue, Bing finished editing my list at item #126 without hallucinations.
Unfortunately, it stopped rephrasing the descriptions after item #18 and output the list as I input it.

Bard Could Not Create Images

Because of the 10,000-character input limitation, I could only submit 99 items from the list in the prompt.
Bard responded, “I can’t create images yet so I’m not able to help you with that.” or “That’s not something I’m able to do yet.”

Ultimately, Claude-instant-100k completed the task with the most accuracy and speed.

You can compare the output from Claude, ChatGPT, and Bing to the original list of items in this Google Sheet.

Task #2: Summarize A 23,500-Word PDF With Claude, ChatGPT, Bing, And Bard

The input: 23,508 words / 125,104 characters copied from a PDF documenting TikTok’s responses to a hearing on how Congress can safeguard United States citizens from the social platform.

The prompt: Please summarize the following:

[the text]

Claude Succincinctly Summarized The Entire Text