Introducing Eleven Realtime: a simpler way to build natural voice agents

Building a voice agent that sounds natural today requires a lot of decisions. You have to make choices on individual components like speech recognition, voice activity detection, turn-taking, the underlying LLM, and text to speech. Building this stack is a project in itself, and getting all the components to work well together takes more testing and tuning.

Eleven Realtime is a new class of Speech to Speech models that removes those hurdles. Each model is a complete voice agent, optimized for separate use cases. Both support more than 90 languages and can use any of the 17,000+ voices in the ElevenLabs library:

Today, we’re launching:

• Eleven Realtime v1 Mini for natural conversations at low cost. This is the best choice for AI assistants, teachers, and interactive characters, and will be sufficient for most use cases.
• Eleven Realtime v1 Max for complex tasks. This is ideal for customer support, lead qualification, and multi-step workflows, where the agent has to follow a process and reason through steps.

Why we built realtime models

You shouldn’t have to become a configuration expert to build a voice agent that sounds natural. Getting an agent to feel and sound right is about small judgment calls that require nuance such as how long to wait after a pause before answering, how to recover when the user interrupts the agent, and how voice, prompts, and a model all work together. In ElevenAgents, these are all settings you can adjust.

The same goes for developers already on a realtime API. Ours is compatible with OpenAI’s Realtime API, so trying Eleven Realtime simply means changing the endpoint and model name in the code you might already have.

How they perform

Artificial Analysis evaluated both models independently across three public benchmarks: Big Bench Audio for speech reasoning, Full-Duplex-Bench for conversational dynamics, and the Speech Agent Arena for human preference and task success.