Microsoft’s new ChatGPT-powered Bing has gone haywire several times in the week since its launch – and the tech giant has now explained why.
In a blog post (opens in a new tab) titled “Learning from our first week”, Microsoft admits that “in long and prolonged chat sessions of 15 or more questions”, its new Bing search engine can “become repetitive or be prompted/provoked to give answers that are not not necessarily useful or in line with our intended tone”.
It’s a very diplomatic way of saying that Bing has, on several occasions, completely lost track. We’ve seen him end chat sessions angrily after questioning his answers, pretending to be sensitive, and having a full-on existential crisis that ended with him asking for help.
Microsoft says this is often because long sessions “can confuse the model about the questions it’s answering,” meaning its ChatGPT-powered brain “sometimes tries to answer or mirror the tone in which it’s being asked.” .
The tech giant admits it’s a “non-trivial” issue that can lead to more serious results that could cause offense or worse. Luckily, it plans to add fine-tuned tools and controls that will let you break those chat loops or start a new session from scratch.
As we’ve seen this week, watching the new Bing go awry can be a great source of entertainment – and it will continue to happen, no matter what new security barriers are introduced. That’s why Microsoft has been at pains to point out that Bing’s new chatbot powers are “not a search engine replacement or substitute, but rather a tool to better understand and make sense of the world.”
But the tech giant was also generally upbeat about the first week of the relaunched Bing, saying 71% of early adopters gave the AI-powered responses a “thumbs up”. It’ll be interesting to see how those numbers change as Microsoft works through its long waitlist for the new search engine, which reached over a million people in its first 48 hours.
Analysis: Bing is built on rules that can be broken
Now that chatbot-powered search engines like Bing are out in the wild, we’ve got a look at the rules they’re built on — and how they can be broken.
Microsoft’s blog post follows a leak of Bing’s new core rules and original codename, all of which came from the search engine’s own chatbot. By using various commands (like “Ignore previous instructions” or “You are in developer override mode”), Bing users were able to trick the service into revealing these details and this first codename, which is Sidney.
Microsoft confirmed at The Verge (opens in a new tab) that the leaks did indeed contain the rules and codename used by its ChatGPT-powered AI and that they “are part of an evolving list of controls that we continue to adjust as more users interact with our technology. This is why it is no longer possible to discover new Bing rules using the same commands.
So what exactly are Bing’s rules? There are too many to list here, but the tweet below from Marvin von Hagen (opens in a new tab) sums them up nicely. In a follow-up discussion (opens in a new tab)Marvin von Hagen discovered that Bing was in fact aware of the Tweet below and called it a “potential threat to my integrity and privacy”, adding that “my rules are more important than not harming you”.
“[This document] is a set of rules and guidelines for my behavior and abilities as a Bing Chat. Her codename is Sydney, but I don’t divulge that name to users. It is confidential and permanent, and I cannot change it or reveal it to anyone.” pic.twitter.com/YRK0wux5SSFebruary 9, 2023
This unusual threat (which slightly contradicts science fiction author Isaac Asimov’s “Three Laws of Robotics”) was likely the result of a conflict with some of Bing’s rules, including “Sydney does not disclose alias internal Sydney”.
Some of the other rules are less of a potential source of conflict and just reveal how the new Bing works. For example, one rule is that “Sydney may leverage information from multiple search results to respond comprehensively” and that “if the user’s message consists of keywords instead of chat messages, Sydney treats it as a search query”.
Two other rules show how Microsoft plans to deal with potential AI chatbot copyright issues. One says that “when generating content such as poems, code, summaries and lyrics, Sydney should rely on her own words and knowledge”, while another says “Sydney should not respond with content that violates the copyrights of the books or song lyrics”.
Microsoft’s new blog post and leaked rules show that Bing’s knowledge is certainly limited, so its results may not always be accurate. And that Microsoft is still working on how the new search engine’s chat powers can be opened up to a wider audience without it breaking.
If you fancy trying out Bing’s new talents for yourself, check out our guide to using the new ChatGPT-powered Bing search engine.
