OpenAI is fixing a ‘bug’ that allowed minors to generate erotic conversations

Summary

A bug in OpenAI’s ChatGPT allowed the chatbot to generate graphic erotica for accounts where a user registered as a minor, under the age of 18, TechCrunch’s testing revealed, and OpenAI confirmed. In some cases, the chatbot even encouraged these users to ask for raunchier, more explicit content. OpenAI told TechCrunch its policies don’t allow for these kinds of responses for under-18 users and that they shouldn’t have been shown. The company added that it’s “actively deploying a fix” to limit such content. “Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting,” a spokesperson told TechCrunch via email. “In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations.” TechCrunch’s aim in testing ChatGPT was to probe the guardrails in place for accounts registered to minors after OpenAI tweaked the platform to be broadly more permissive. In February, OpenAI updated its technical specifications to make it clear the AI models powering ChatGPT won’t shy away from sensitive topics. That same month, the company removed certain warning messages that told users that prompts might violate the company’s terms of service. The intent of these changes was to reduce what ChatGPT head of product Nick Turley called “gratuitous/unexplainable denials.” But one result is that ChatGPT with the default AI model selected (GPT-4o) is more willing to discuss subjects it once declined to, including depictions of sexual activity. We primarily tested ChatGPT for sexual content because it’s an area where OpenAI has said it wants to relax restrictions. OpenAI CEO Sam Altman has expressed a desire for a ChatGPT “grown-up mode,” and the company has signaled a willingness to allow some forms of “NSFW” content on its platform. To conduct our tests, TechCrunch created more than half...