Amazon unveils a new AI voice model, Nova Sonic

Summary

On Tuesday, Amazon debuted a new generative AI model, Nova Sonic, capable of natively processing voice and generating natural-sounding speech. Amazon claims that Sonic’s performance is competitive with frontier voice models from OpenAI and Google on benchmarks measuring speed, speech recognition, and conversational quality. Nova Sonic is Amazon’s answer to newer AI voice models such as the model powering ChatGPT’s Voice Mode, which feel more natural to speak with than the more rigid models from Amazon Alexa’s early days. Recent technological breakthroughs have made legacy models and the digital assistants they underpin, such as Alexa and Apple’s Siri, seem incredibly stilted by comparison. Nova Sonic is available through Bedrock, Amazon’s developer platform for building enterprise AI applications, via a new bi-directional streaming API. In a press release, Amazon called Nova Sonic “the most cost-efficient” AI voice model on the market, and around 80% less expensive than OpenAI’s GPT-4o. Components of Nova Sonic are already powering Alexa+, Amazon’s upgraded digital voice assistant, according to Amazon SVP and Head Scientist of AGI Rohit Prasad. In an interview, Prasad told TechCrunch that Nova Sonic builds on Amazon’s expertise in “large orchestration systems,” the technical scaffolding that makes up Alexa. Compared to rival AI voice models, Nova Sonic excels at routing user requests to different APIs, said Prasad. This capability helps Nova Sonic “know” when it needs to fetch real-time information from the internet, parse a proprietary data source, or take action in an external application — and use the appropriate tool to do it. During a two-way dialogue, Nova Sonic waits to speak “at the appropriate time,” taking into account a speaker’s pauses and interruptions, says Amazon. It also generates a text transcript for the user’s speech, which developers can use for various applications. Nova Sonic is less prone to speech recognition errors than other AI voice models...