ElevenLabs Realtime Voice Agent: Shipping Speech-to-Speech Without a Rewrite
Voice products live and die on the pause between your user finishing a sentence and the agent starting its reply. ElevenLabs' Realtime voice agent platform—highlighted on Product Hunt around October 23, 2026 with roughly 185 upvotes—targets that pause directly: speech-to-speech models with about 570ms to first audio, exposed through an OpenAI-compatible Realtime API. This guide covers what to verify before you build on it.
What shipped: a fast STS voice agent with a familiar API
The Realtime platform at elevenlabs.io/realtime takes audio in and produces audio out, with the model handling understanding, reasoning, and speech generation in one pass. Launch benchmarks circulated roughly 570ms to first audio—fast enough that conversations feel human-paced rather than walkie-talkie-paced.
The strategic detail is protocol compatibility. The endpoint speaks the OpenAI Realtime protocol, which means the growing body of clients, SDKs, and patterns built for OpenAI's realtime surface can be repointed at ElevenLabs. For teams, that converts a vendor migration from a project into a config change.
One clarification that matters in October 2026 coverage: Realtime is not Chimera. Chimera is a separate ElevenLabs model effort; Realtime is the voice-agent platform. Search interest has been blending the two.
Where the milliseconds actually go
A published "~570ms first audio" is a lab number under ideal conditions. Your production loop adds:
- Transport. WebSocket establishment, mobile network jitter, and regional routing to the inference endpoint.
- Turn policy. How long your client waits to decide the user actually finished—the end-of-turn detector trades interruption risk for response delay.
- Tool calls. If the agent must look up an order mid-conversation, that round-trip is part of perceived latency.
- Escalation paths. Handing off to a human or a bigger model mid-call.
The evaluation habit that pays off: record 50 real transcripts, replay them against every candidate provider, and measure p50/p95 first-audio plus interruption recovery. For a deeper treatment—including the v1 Mini versus v1 Max tier choice—see our companion guide on ElevenLabs Realtime voice agents.
The rest of the voice agent stack
The model is one component. Pieces AIXList tracks that pair well with a Realtime deployment:
- Context Engine — shared memory so the voice agent knows what the customer already told your text agents.
- PhoneUse — cloud phones when the voice agent must also act inside mobile apps.
- Block Dessa speech-to-speech — where the architecture is heading: audio-only training and local inference.
- Monish — approval flows for the agents your voice agent orchestrates.
Migration checklist: OpenAI Realtime to ElevenLabs
- Repoint the client at the ElevenLabs endpoint; confirm authentication and streaming behavior match.
- Re-test every language you serve—coverage and accent performance differ between model families.
- Verify tool-calling (bookings, lookups, refunds) works mid-conversation, not just in text-only tests.
- Price a full month of projected audio minutes including retries, escalations, and abandoned calls.
- Keep a vendor abstraction on the WebSocket client so the next switch is also a config change.
ElevenLabs Realtime voice agent FAQ
What is the ElevenLabs Realtime voice agent?
ElevenLabs Realtime is a speech-to-speech voice agent platform that takes audio in and returns audio out—first audio in roughly 570ms in launch benchmarks—behind an OpenAI-compatible Realtime API at elevenlabs.io/realtime.
How fast is ElevenLabs Realtime?
Launch coverage cited approximately 570ms to first audio. Your real numbers depend on language, network path, and turn-taking policy—benchmark on your own call transcripts before committing to latency-sensitive flows.
Can I migrate from OpenAI Realtime to ElevenLabs Realtime?
Yes—ElevenLabs ships an OpenAI-compatible Realtime endpoint, so clients built on the OpenAI Realtime protocol can be repointed at ElevenLabs to compare quality and cost without rewriting the integration.
Is ElevenLabs Realtime the same as Chimera?
No. Realtime is the speech-to-speech voice agent platform (STS models behind the Realtime API); Chimera is a separate ElevenLabs model effort. Confusing the two is common in October 2026 coverage.
What do I still need to build around a Realtime voice agent?
The model handles conversation, but you still own telephony or in-app transport, tool integrations, memory across calls, and evaluation. Tools like Context Engine for shared memory and PhoneUse for phone-app actions fill those gaps.
AIXList is an independent AI tools directory. This article summarizes public October 2026 announcements and launch coverage; we are not affiliated with ElevenLabs or OpenAI. Latency and pricing figures are launch-time claims—verify against official documentation before committing.