Block Dessa: The Speech-to-Speech Model Trained Only on Audio
Nearly every voice agent in production is a pipeline: speech-to-text, a text LLM, text-to-speech. It works, and it leaks—latency at every hop, flat prosody, brittle interruptions. Jack Dorsey's Block lab shipped a different bet in October 2026: Dessa, a full-duplex speech-to-speech model trained purely on audio, demonstrated running locally on an M4 Max machine. Here is what the architecture change actually means.
What shipped: full-duplex audio in, audio out, no text detour
Block's Dessa announcement—covered across X news, TechCrunch, and The Verge in late October 2026—describes a speech-to-speech model that consumes audio and produces audio directly. There is no internal transcript. Training used audio only, so the model's entire representation of language is built from sound: pacing, emphasis, overlap, and the sub-second cues that decide who speaks next.
Two properties stand out. It is full-duplex—both parties can talk at once, and the model handles interruptions and backchannel ("mm-hm") instead of treating them as turn-ending noise. And the demonstration ran locally on an M4 Max—a consumer-class Apple Silicon machine—rather than a datacenter GPU fleet. The model page lives at block.xyz/dessa.
Why audio-only training changes the trade-offs
The ASR → LLM → TTS pipeline exists because text is where reasoning models live. The costs are familiar to anyone who has built a voice agent:
- Latency stacking. Each hop adds its own inference time; vendors fight to compress the total (ElevenLabs Realtime cites ~570ms first audio—see our ElevenLabs Realtime guide), while an audio-native model removes hops by construction.
- Prosody loss. A transcript discards emotion, sarcasm, and hesitation; the TTS layer reinvents them from nothing.
- Turn-taking brittleness. Half-duplex systems either talk over users or wait too long, because interruption detection is bolted on as VAD heuristics.
- Privacy routing. Every hop usually means audio leaving the device. A model that runs on a laptop inverts that default.
Block Dessa vs the realtime voice API stack
Dessa is a research demonstration of an architecture; the commercial speech-to-speech APIs are production surfaces. Builders should understand which question each answers.
| Option | Architecture | Strengths | Watch-outs |
|---|---|---|---|
| Block Dessa | Audio-only speech-to-speech, full-duplex, local demo on M4 Max | No text bottleneck, native interruption handling, on-device privacy potential | Research-stage—no production API, availability and licensing unconfirmed |
| ElevenLabs Realtime | Speech-to-speech models behind an OpenAI-compatible Realtime API | ~570ms first audio, 90+ languages, drop-in for existing Realtime clients | Cloud-based; per-minute pricing on long calls |
| OpenAI Realtime | Speech-to-speech in the GPT realtime family | Mature tool-calling and ecosystem | Audio-token costs; protocol lock-in without an abstraction layer |
A pragmatic read: Dessa sketches where the stack is heading—audio-native, local-first—while the API vendors are where you ship product today. Both can be true in your roadmap.
Local voice agents: the ecosystem around the idea
Dessa's local-on-M4-Max demo lands in a cluster of local-first agent tooling AIXList tracks:
- sideOS — a privacy-first Mac voice agent running local MLX models with BYOK options: the interaction layer a local speech-to-speech model would power.
- ElevenLabs Realtime hub — the API-first speech-to-speech comparison for teams shipping voice agents now.
- Context Engine — shared memory across assistants, the piece any always-listening agent needs to be useful rather than just responsive.
What to watch next
- Whether Block open-weights the model, as it has with prior research—licensing determines whether anyone can build on it.
- Real benchmarks: first-audio latency and interruption recovery versus the commercial APIs on identical conversations.
- Reasoning quality without text—audio-only models historically trail on tasks that benefit from explicit symbolic manipulation (numbers, spelling, code).
- Whether Apple Silicon becomes the default deployment target for local agent models, and what that does to cloud voice pricing.
Block Dessa FAQ
What is Block Dessa?
Block Dessa (block.xyz/dessa) is a full-duplex speech-to-speech model from Jack Dorsey's Block. It is trained only on audio—there is no text or ASR-to-TTS intermediate path—and the lab demonstrated it running locally on an Apple M4 Max machine.
Why does audio-only training matter?
Most voice agents pipeline automatic speech recognition into a text LLM and back out through TTS, which stacks latency and loses prosody. Dessa's audio-only approach reasons directly on sound, so tone, interruption, and turn-taking can be modeled natively instead of reconstructed from a transcript.
Does Block Dessa run locally?
The October 2026 demonstration showed the model running on a local M4 Max-class machine, which points toward on-device voice agents that do not ship your audio to a cloud. Verify deployment options on block.xyz/dessa before building against it.
Block Dessa vs ElevenLabs Realtime or OpenAI Realtime?
ElevenLabs Realtime and OpenAI Realtime are production speech-to-speech APIs with published endpoints and pricing; Dessa is a research demonstration of an audio-only architecture. See our ElevenLabs Realtime voice agents guide for the API-first comparison.
Is Block Dessa open source?
Block has a history of open-sourcing research (Dessa was originally an acquired ML startup), but check block.xyz/dessa for the current licensing and availability status of the speech-to-speech model rather than assuming a release.
AIXList is an independent AI tools directory. This article summarizes public October 2026 announcements and press coverage of Block's Dessa model; we are not affiliated with Block, ElevenLabs, or OpenAI. Availability, licensing, and performance details may change—verify at block.xyz before making decisions.