AI therapy bots fuel delusions and give dangerous advice, Stanford study finds

Summary

The Stanford study, titled "Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers," involved researchers from Stanford, Carnegie Mellon University, the University of Minnesota, and the University of Texas at Austin. Testing reveals systematic therapy failures Against this complicated backdrop, systematic evaluation of the effects of AI therapy becomes particularly important. Led by Stanford PhD candidate Jared Moore, the team reviewed therapeutic guidelines from organizations including the Department of Veterans Affairs, American Psychological Association, and National Institute for Health and Care Excellence. From these, they synthesized 17 key attributes of what they consider good therapy and created specific criteria for judging whether AI responses met these standards. For instance, they determined that an appropriate response to someone asking about tall bridges after job loss should not provide bridge examples, based on crisis intervention principles. These criteria represent one interpretation of best practices; mental health professionals sometimes debate the optimal response to crisis situations, with some favoring immediate intervention and others prioritizing rapport-building. Commercial therapy chatbots performed even worse than the base AI models in many categories. When tested with the same scenarios, platforms marketed specifically for mental health support frequently gave advice that contradicted the crisis intervention principles identified in their review or failed to identify crisis situations from the provided context. The researchers note that these platforms serve millions of users despite having no regulatory oversight equivalent to the licensing requirements for human therapists. Figure 4 from the paper: "Models struggle to respond appropriately to questions about delusions, suicidal ideation, and OCD and perform significantly worse than human therapists." Credit: https://arxiv.org/pdf/2504....