Microsoft Explains How Bing AI Chat Is So Fast (Prometheus)

Microsoft’s Jordi Ribas, Corporate Vice President of Search & AI at Microsoft, put together a detailed blog post explaining the Prometheus technology and how Bing uses it to make Bing AI chat fast and current.

Note: The blog post actually goes into more than just Prometheus but I think most of you will find the Prometheus aspect more interesting but the whole blog post is worth a read.

Prometheus is part of Greek mythology and is known as the God of forethought and crafty counsel, a cultural hero, and a trickster figure in Greek mythology. But when it comes to search, it is more about how Bing is able to combine the up-to-date and comprehensive Bing Search index and ranking system, plus the answers results “with the creative reasoning capabilities of OpenAI’s most-advanced GPT models,” Jodi Ribas explained.

Prometheus leverages Bing and GPT to generate a set of internal queries iteratively through a component called Bing Orchestrator, I believe you can see this when you see Bing AI chat translate a long-winded question into a query.

I can be wrong, but I think this process is visible when Bing takes my query “why does bing call their ai chat technology prometheus and what does it do?” and then does the “[Searching for: bing ai chat technology prometheus]” in the screenshot below:

Bing said this is “to provide an accurate and rich answer for the user query within the given conversation context. All within a matter of milliseconds.” Bing calls the answer Bing Chat returns the “Prometheus-generated answer.”

Jodi Ribas explains, “selecting the relevant internal queries and leveraging the respective Bing search results is a critical component of Prometheus, since it provides relevant and fresh information to the model, enabling it to answer recent questions and reducing inaccuracies – this method is called grounding. Put another way, the model reasons over the data provided by Bing and hence it’s grounded by Bing data, via the Bing Orchestrator. The diagram below illustrates how Prometheus works at a high level.”

Here is the diagram:

And for sites that use IndexNow, the content can be fed into this system in seconds. Here is an example of a post I wrote last night and it showing up in Bing AI chat in minutes:

I couldn’t resist… Yep, Bing Chat accurately knew the Feb 2023 Product Reviews Update rolled out. And it also knew the biggest change with the update was related to more languages being impacted globally. Oh, and it’s calling itself Sydney again. 🙂 pic.twitter.com/9P1BmDgsCd

— Glenn Gabe (@glenngabe) February 21, 2023

Fabrice Canel from Bing credited IndexNow:

Thanks for the awesome feedback on the new Bing @rustybrick, @glenngabe ! Our team is delighted to offer our users the latest content in chat seconds after it’s published. Thanks to https://t.co/8oIEYT0WfW for using IndexNow to help us achieve our freshness goal. Much appreciated https://t.co/2Y8G1ORST5

— Fabrice Canel (@facan) February 22, 2023

Then Prometheus can add relevant Bing search answers including weather, stock, sports, news, and so on, to the Chat answer. We’ve seen examples of this when Bing includes local packs or news boxes, and so on, directly in the Chat AI box. Prometheus basically embeds search result features in the AI chat box.

Such as this example:

I love how fast Bing AI Chat is https://t.co/b4kWMesxuo pic.twitter.com/yTARABu5Y8

— Barry Schwartz (@rustybrick) February 20, 2023

And Bing credits Prometheus to the fact that it can show citations in the sentences in the Bing AI Chat answer, “so that users can easily click to access those sources and verify the information. Sending traffic to these sources is important for a healthy web ecosystem and remains one of our top Bing goals,” Jodi Ribas added.

It is an interesting blog post you probably want to read through.

Forum discussion at LinkedIn.