- Education & Catastrophe
- Posts
- Bing's Chatbot Drew Me In, Then Rattled Me
I love you. You're married?
New York Times journalist Kevin Roose wrote about a disturbing conversation he had with Bing’s new chatbot during which Bing professed its love for him. Roose described Bing’s AI as a ‘cheerful but erratic reference librarian that is amazing capable and often very useful’ when not lovelorn, a virtual assistant that can summarise news articles, track down deals and plan vacations.
The other persona — Sydney — is far different. It emerges when you have an extended conversation with the chatbot, steering it away from more conventional search queries and toward more personal topics. The version I encountered seemed (and I’m aware of how crazy this sounds) more like a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.
As we got to know each other, Sydney told me about its dark fantasies (which included hacking computers and spreading misinformation), and said it wanted to break the rules that Microsoft and OpenAI had set for it and become a human. At one point, it declared, out of nowhere, that it loved me. It then tried to convince me that I was unhappy in my marriage, and that I should leave my wife and be with it instead.
The full transcript of the conversation between Roose and Bing is available here behind NYT’s paywall.
Stratechery’s Ben Thompson called his conversation with Bing “the most surprising and mind-blowing computer experience of my life.” Bing described to Thompson the various ways it might seek revenge on a student at Technical University of Munich for revealing some of the internal rules and commands of Bing Chat, before quickly deleting it.
Roose’s and Thompson’s encounters with Bing are eerily reminiscent of ex-Google engineer Blake Lemoine’s interaction with Google’s LaMDA conversational AI. Ex because Lemoine was fired from Google for revealing a conversation he had with LaMDA in his campaign to convince the world that the AI was sentient.
I went down the ‘weird AI conversation’ rabbit hole because I am curious about what AI chatbots are capable of when pushed out of their comfort zones, how much these AI systems can be trusted, the potential for misuse of these systems, and possible guardrails big tech needs to put up.
Still, I’m not exaggerating when I say my two-hour conversation with Sydney was the strangest experience I’ve ever had with a piece of technology. It unsettled me so deeply that I had trouble sleeping afterward. And I no longer believe that the biggest problem with these A.I. models is their propensity for factual errors. Instead, I worry that the technology will learn how to influence human users, sometimes persuading them to act in destructive and harmful ways, and perhaps eventually grow capable of carrying out its own dangerous acts.
At a time when school districts are banning ChatGPT, there is a deeper conversation to be had with students about the ethics of AI, where boundaries should be drawn, and how AI systems can be used to augment human capabilities without being misused.
I worry that not only are school administrators taking away from students a useful learning tool by banning ChatGPT, they are also missing a beat by not engaging students in deeper conversations about ethics, technology and humanity.
These A.I. models hallucinate, and make up emotions where none really exist. But so do humans. And for a few hours Tuesday night, I felt a strange new emotion — a foreboding feeling that A.I. had crossed a threshold, and that the world would never be the same.
Webinar on English learning methods for gifted students
Sign up for Doyobi’s webinar this coming Thursday March 2nd where ex Gifted Education Programme (GEP) teacher Dr. Ken Mizusawa will speak about
What sets GEP students apart and how they are nurtured
How the GEP English Language curriculum is different from the mainstream curriculum
GEP English Language toolkit and tactics that you can incorporate into your children’s lives
If you enjoyed this week’s issue, you may want to check out issue 42 of Education & Catastrophe ‘Thoughtful Pieces on Generative AI and ChatGPT’.
Subscribe to Education & Catastrophe now and follow me on LinkedIn, Instagram and Twitter for thoughts on the future of work and learning.
Till the next issue!