Bing Also Flubs Its Demos ⇥ dkb.blog

After I linked to an example of a confident but wrong answer from Google Bard, reader Nick Jarman emailed me with a tip about a similar flub in Microsoft’s Bing demo queries.

If you follow the prompt suggested on the New Bing homepage for making a pop music trivia game, the fifth suggested question is about an artist’s discography:

5. Which pop artist released the albums Future Nostalgia, Confessions on a Dance Floor, and Chromatica?

A) Dua Lipa

B) Madonna

C) Lady Gaga

D) Kylie Minogue

Bing says “A” is correct, but this question is unanswerable: “Future Nostalgia” is by Dua Lip, “Confessions…” is by Madonna, and “Chromatica” is by Lady Gaga. And that is not the only showpiece query where Bing gets confused.

Dmitri Brereton documents several other examples in Bing’s demo materials:

I am shocked that the Bing team created this pre-recorded demo filled with inaccurate information, and confidently presented it to the world as if it were good.

I am even more shocked that this trick worked, and everyone jumped on the Bing AI hype train without doing an ounce of due diligence.

Bing AI is incapable of extracting accurate numbers from a document, and confidently makes up information even when it claims to have sources.

I am not surprised these products are getting things wrong — they are brand new, after all — but it is disappointing to see these kinds of problems in the demos and marketing materials from these two giant companies. A demo is usually showing a company’s products in their best light and the limitations are discovered elsewhere. Is this the best they can do? Credit for the honesty, however unintended, but it only exposes the gap between the excited marketing of these nascent products and their reality.

Thanks again to Jarman for tipping me off to that first flawed demo.