Boson AI's Higgs Realtime Duplex: The Voice AI That Listens, Talks, Thinks and Works at Once

Introduction: Beyond the Walkie-Talkie Voice Assistant

Almost every "conversational" AI assistant you've used is a walkie-talkie: you talk, it goes quiet, you wait, it answers. On November 5, Boson AI introduced Higgs Realtime Duplex — its first full-duplex interactive omni model — and the pitch is that the walkie-talkie era is over. Like a person on a video call, it listens, watches and talks at the same time, shows a face of its own rendered in real time, and — while doing all of that — thinks, calls tools and works on tasks in the background without pausing the conversation.

For anyone building with voice AI tools, this is a meaningful step. Voice platforms like Vapi and voice model providers like ElevenLabs have made phone agents and assistants practical, but the interaction model is still mostly turn-based: transcribe, think, respond. Duplex collapses those turns into a continuous stream — which changes both what the experience feels like and what the underlying infrastructure has to do.

What Full Duplex Actually Means Here

The behavioral details are what make Duplex feel less like an API and more like a participant:

• It decides when to speak — saying "mm-hm" while you talk, answering before you finish a sentence, and stopping the instant you cut in.
• It ignores what isn't addressed to it — background noise and side remarks meant for someone else don't trigger a response.
• It keeps watching — your camera or shared screen streams in continuously, even during silence, so visual context is never lost mid-conversation.
• It has a face — an avatar rendered in real time, synchronized with its voice, so the output channel is audio and visual.

Turn-taking, barge-in and backchannel acknowledgments are the hard, human parts of conversation that turn-based pipelines fake with voice-activity detection. Duplex handles them natively, in the model itself.

The Architecture: 160 ms Chunks and a 45B MoE

Under the hood, the design is elegantly brutal: every stream is sliced into 160-millisecond chunks — your voice and camera feed coming in; the model's voice, face and background work going out. At each step the model reads the chunk that just arrived and writes the chunk that plays next. Silence is represented as chunks too, so the model never stops processing the scene.

The front model is a 45B-parameter Mixture-of-Experts trained on roughly 2 trillion tokens of text, audio and video. Boson explicitly tested larger models — they scored higher but cost more to run. The trick is that the hardest questions can be delegated to a bigger backend model asynchronously, so a mid-sized front model with solid intelligence wins on the cost-performance curve.

Full duplex fills context fast, so Boson compresses aggressively: one audio token per chunk in, two out, a camera frame only every six chunks, and two latents for the face. All in, the audio and visual streams cost about 2.6K tokens per minute, rising to roughly 2.6–5K tokens per minute with thinking and tool calls in the mix.

It Works While It Talks

The most consequential feature for agent builders is background work. Thinking and tool-call generation can span multiple chunks; tool results enter the input stream as payload chunks as they arrive. Some tools return intermediate results in stages — and the model starts speaking about partial results without waiting for the full output, keeping the conversation alive until everything lands.

That's a different programming model from the function-calling loops most voice agent stacks use today. Instead of "call tool, freeze, resume," you get an agent that narrates its own work in progress — much closer to a colleague screen-sharing than a phone tree.

How It Benchmarks

Boson says it compared Higgs Realtime Duplex against the Thinking Machines Lab interaction model and the realtime and live models available from OpenAI and Google, across a suite covering:

• Spoken question answering and full-duplex interaction
• Joint video and audio understanding
• Proactive video — the model watches a clip and chooses when to speak
• Agentic task completion and text instruction following

The company's framing is that Duplex is "built to feel human, to be genuinely smart, and to cost little," with the mid-sized front model plus delegation as the cost lever. As always with vendor-run evaluations, treat the rankings as directional until third-party numbers appear — but the design choices (chunked omni streaming, async delegation) are verifiable in the published architecture regardless of who wins the benchmark.

Pricing and Availability

Higgs Realtime Duplex itself is not yet generally available — an API is "coming soon," with a public waitlist. What you can use today is its sibling, Higgs Realtime, the end-to-end speech-to-speech model with avatar output:

That OpenAI-compatible wire protocol matters: most existing realtime integrations migrate with config changes, so trying Higgs alongside your current stack is an afternoon of work, not a rebuild.

What It Means for Your Voice AI Tool Stack

Three practical takeaways if you build voice agents or interactive assistants:

• Interaction quality becomes a model property, not a pipeline hack. If turn detection, barge-in and "was that addressed to me?" live in the model, you can delete a lot of VAD and orchestration glue — the same consolidation speech-native tools like TomorrowVoice and transcription layers like Whisper-based pipelines have been chipping away at from the other end.
• Background tool-calling changes UX expectations. Users trained on Duplex-style agents will expect progress narration ("I'm pulling that up now…") rather than hold music. Design your function schemas to return staged results.
• Delegation is the new scaling pattern. A cheap always-on front model with an expensive model on call is an architecture you can adopt today inside platforms like Vapi — route easy turns to a small model and escalate only when confidence drops.

If voice output is your bottleneck rather than interaction, avatar video tools like HeyGen and Synthesia cover the pre-recorded end; Duplex is aimed squarely at the live, unscripted end of that spectrum.

Limitations to Keep in Mind

• No public API yet. Duplex is waitlist-only; production builders should benchmark Higgs Realtime today and plan the Duplex migration later.
• Context fills fast. Processing every 160 ms chunk — even during silence — means long sessions need the aggressive stream compression Boson describes, and very long conversations will still hit context limits.
• A rendered face raises the stakes. Once an assistant has a face, users read emotional intent into it; teams deploying avatars take on perception and uncanny-valley problems that voice-only agents avoid.
• Vendor benchmarks only. The comparison suite is Boson's own; independent evaluations haven't landed yet.

Frequently Asked Questions