← Back to Resources
Voice modelsOctober 25, 2026

Block Dessa: The Speech-to-Speech Model Trained Only on Audio

Nearly every voice agent in production is a pipeline: speech-to-text, a text LLM, text-to-speech. It works, and it leaks—latency at every hop, flat prosody, brittle interruptions. Jack Dorsey's Block lab shipped a different bet in October 2026: Dessa, a full-duplex speech-to-speech model trained purely on audio, demonstrated running locally on an M4 Max machine. Here is what the architecture change actually means.

What shipped: full-duplex audio in, audio out, no text detour

Block's Dessa announcement—covered across X news, TechCrunch, and The Verge in late October 2026—describes a speech-to-speech model that consumes audio and produces audio directly. There is no internal transcript. Training used audio only, so the model's entire representation of language is built from sound: pacing, emphasis, overlap, and the sub-second cues that decide who speaks next.

Two properties stand out. It is full-duplex—both parties can talk at once, and the model handles interruptions and backchannel ("mm-hm") instead of treating them as turn-ending noise. And the demonstration ran locally on an M4 Max—a consumer-class Apple Silicon machine—rather than a datacenter GPU fleet. The model page lives at block.xyz/dessa.

Why audio-only training changes the trade-offs

The ASR → LLM → TTS pipeline exists because text is where reasoning models live. The costs are familiar to anyone who has built a voice agent:

  • Latency stacking. Each hop adds its own inference time; vendors fight to compress the total (ElevenLabs Realtime cites ~570ms first audio—see our ElevenLabs Realtime guide), while an audio-native model removes hops by construction.
  • Prosody loss. A transcript discards emotion, sarcasm, and hesitation; the TTS layer reinvents them from nothing.
  • Turn-taking brittleness. Half-duplex systems either talk over users or wait too long, because interruption detection is bolted on as VAD heuristics.
  • Privacy routing. Every hop usually means audio leaving the device. A model that runs on a laptop inverts that default.

Block Dessa vs the realtime voice API stack

Dessa is a research demonstration of an architecture; the commercial speech-to-speech APIs are production surfaces. Builders should understand which question each answers.

OptionArchitectureStrengthsWatch-outs
Block DessaAudio-only speech-to-speech, full-duplex, local demo on M4 MaxNo text bottleneck, native interruption handling, on-device privacy potentialResearch-stage—no production API, availability and licensing unconfirmed
ElevenLabs RealtimeSpeech-to-speech models behind an OpenAI-compatible Realtime API~570ms first audio, 90+ languages, drop-in for existing Realtime clientsCloud-based; per-minute pricing on long calls
OpenAI RealtimeSpeech-to-speech in the GPT realtime familyMature tool-calling and ecosystemAudio-token costs; protocol lock-in without an abstraction layer

A pragmatic read: Dessa sketches where the stack is heading—audio-native, local-first—while the API vendors are where you ship product today. Both can be true in your roadmap.

Local voice agents: the ecosystem around the idea

Dessa's local-on-M4-Max demo lands in a cluster of local-first agent tooling AIXList tracks:

  • sideOS — a privacy-first Mac voice agent running local MLX models with BYOK options: the interaction layer a local speech-to-speech model would power.
  • ElevenLabs Realtime hub — the API-first speech-to-speech comparison for teams shipping voice agents now.
  • Context Engine — shared memory across assistants, the piece any always-listening agent needs to be useful rather than just responsive.

What to watch next

  • Whether Block open-weights the model, as it has with prior research—licensing determines whether anyone can build on it.
  • Real benchmarks: first-audio latency and interruption recovery versus the commercial APIs on identical conversations.
  • Reasoning quality without text—audio-only models historically trail on tasks that benefit from explicit symbolic manipulation (numbers, spelling, code).
  • Whether Apple Silicon becomes the default deployment target for local agent models, and what that does to cloud voice pricing.

Block Dessa FAQ

What is Block Dessa?

Block Dessa (block.xyz/dessa) is a full-duplex speech-to-speech model from Jack Dorsey's Block. It is trained only on audio—there is no text or ASR-to-TTS intermediate path—and the lab demonstrated it running locally on an Apple M4 Max machine.

Why does audio-only training matter?

Most voice agents pipeline automatic speech recognition into a text LLM and back out through TTS, which stacks latency and loses prosody. Dessa's audio-only approach reasons directly on sound, so tone, interruption, and turn-taking can be modeled natively instead of reconstructed from a transcript.

Does Block Dessa run locally?

The October 2026 demonstration showed the model running on a local M4 Max-class machine, which points toward on-device voice agents that do not ship your audio to a cloud. Verify deployment options on block.xyz/dessa before building against it.

Block Dessa vs ElevenLabs Realtime or OpenAI Realtime?

ElevenLabs Realtime and OpenAI Realtime are production speech-to-speech APIs with published endpoints and pricing; Dessa is a research demonstration of an audio-only architecture. See our ElevenLabs Realtime voice agents guide for the API-first comparison.

Is Block Dessa open source?

Block has a history of open-sourcing research (Dessa was originally an acquired ML startup), but check block.xyz/dessa for the current licensing and availability status of the speech-to-speech model rather than assuming a release.

AIXList is an independent AI tools directory. This article summarizes public October 2026 announcements and press coverage of Block's Dessa model; we are not affiliated with Block, ElevenLabs, or OpenAI. Availability, licensing, and performance details may change—verify at block.xyz before making decisions.