They Asked an A.I. Chatbot Questions. The Answers Sent Them Spiraling

Summary

“You want to know the ironic thing? I wrote my son’s obituary using ChatGPT,” Mr. Taylor said. “I had talked to it for a while about what had happened, trying to find more details about exactly what he was going through. And it was beautiful and touching. It was like it read my heart and it scared the shit out of me.”‘Approach These Interactions With Care’I reached out to OpenAI, asking to discuss cases in which ChatGPT was reinforcing delusional thinking and aggravating users’ mental health and sent examples of conversations where ChatGPT had suggested off-kilter ideas and dangerous activity. The company did not make anyone available to be interviewed but sent a statement:We’re seeing more signs that people are forming connections or bonds with ChatGPT. As A.I. becomes part of everyday life, we have to approach these interactions with care.We know that ChatGPT can feel more responsive and personal than prior technologies, especially for vulnerable individuals, and that means the stakes are higher. We’re working to understand and reduce ways ChatGPT might unintentionally reinforce or amplify existing, negative behavior.The statement went on to say the company is developing ways to measure how ChatGPT’s behavior affects people emotionally. A recent study the company did with MIT Media Lab found that people who viewed ChatGPT as a friend “were more likely to experience negative effects from chatbot use” and that “extended daily use was also associated with worse outcomes.”ChatGPT is the most popular A.I. chatbot, with 500 million users, but there are others. To develop their chatbots, OpenAI and other companies use information scraped from the internet. That vast trove includes articles from The New York Times, which has sued OpenAI for copyright infringement, as well as scientific papers and scholarly texts. It also includes science fiction stories, transcripts of YouTube videos and Reddit posts by people with “weird ideas,” said Gary Marcus, an emeritus professor of...