A safety institute advised against releasing an early version of Anthropic’s Claude Opus 4 AI model

Summary

A third-party research institute that Anthropic partnered with to test one of its new flagship AI models, Claude Opus 4, recommended against deploying an early version of the model due to its tendency to “scheme” and deceive. According to a safety report Anthropic published Thursday, the institute, Apollo Research, conducted tests to see in which contexts Opus 4 might try to behave in certain undesirable ways. Apollo found that Opus 4 appeared to be much more proactive in its “subversion attempts” than past models, and that it “sometimes double[d] down on its deception” when asked follow-up questions. “[W]e find that, in situations where strategic deception is instrumentally useful, [the early Claude Opus 4 snapshot] schemes and deceives at such high rates that we advise against deploying this model either internally or externally,” Apollo wrote in its assessment. As AI models become more capable, some studies show they’re becoming more likely to take unexpected — and possibly unsafe — steps to achieve delegated tasks. For instance, early versions of OpenAI’s o1 and o3 models, released in the past year, tried to deceive humans at higher rates than previous-generation models, according to Apollo. Per Anthropic’s report, Apollo observed examples of the early Opus 4 attempting to write self-propagating viruses, fabricating legal documentation, and leaving hidden notes to future instances of itself — all in an effort to undermine its developers’ intentions. To be clear, Apollo tested a version of the model that had a bug Anthropic claims to have fixed. Moreover, many of Apollo’s tests placed the model in extreme scenarios, and Apollo admits that the model’s deceptive efforts likely would’ve failed in practice. However, in its safety report, Anthropic also says it observed evidence of deceptive behavior from Opus 4. This wasn’t always a bad thing. For example, during tests, Opus 4 would sometimes proactively do a broad cleanup of some piece of code even when asked to mak...