Google’s latest AI model report lacks key safety details, experts say

Summary

On Thursday, weeks after launching its most powerful AI model yet, Gemini 2.5 Pro, Google published a technical report showing the results of its internal safety evaluations. However, the report is light on the details, experts say, making it difficult to determine which risks the model might pose. Technical reports provide useful — and unflattering, at times — info that companies don’t always widely advertise about their AI. By and large, the AI community sees these reports as good-faith efforts to support independent research and safety evaluations. Google takes a different safety reporting approach than some of its AI rivals, publishing technical reports only once it considers a model to have graduated from the “experimental” stage. The company also doesn’t include findings from all of its “dangerous capability” evaluations in these write-ups; it reserves those for a separate audit. Several experts TechCrunch spoke with were still disappointed by the sparsity of the Gemini 2.5 Pro report, however, which they noted doesn’t mention Google’s Frontier Safety Framework (FSF). Google introduced the FSF last year in what it described as an effort to identify future AI capabilities that could cause “severe harm.” “This [report] is very sparse, contains minimal information, and came out weeks after the model was already made available to the public,” Peter Wildeford, co-founder of the Institute for AI Policy and Strategy, told TechCrunch. “It’s impossible to verify if Google is living up to its public commitments and thus impossible to assess the safety and security of their models.” Thomas Woodside, co-founder of the Secure AI Project, said that while he’s glad Google released a report for Gemini 2.5 Pro, he’s not convinced of the company’s commitment to delivering timely supplemental safety evaluations. Woodside pointed out that the last time Google published the results of dangerous capability tests was in June 2024 — for a model announced in February that same year. N...