Diffusion LLMs Hit the Big Time: Mercury 3 Generates 1,000+ Tokens a Second

The Release: Mercury 3 Sets a New Speed-Intelligence Pareto Frontier

For most of the LLM era, one assumption went unchallenged: smarter models are slower models. You picked your point on the curve — a frontier model that thinks for thirty seconds, or a speed-optimized one that answers instantly but stumbles on hard tasks. On October 19, Redwood City–based Inception Labs launched Mercury 3, and its central claim is that this trade-off just collapsed: the most capable diffusion large language model (dLLM) ever released beats speed-optimized frontier models — GPT-6 Luna and Claude Haiku 5.5 — across a range of benchmarks while running 4–8x faster, sustaining more than 1,000 tokens per second on widely available NVIDIA GPUs.

The scale of the jump is unusual. Mercury 3 is more than 40 times the size of Mercury 1, Inception's first commercial dLLM from June 2025, yet inference speeds stayed comparable. Each release in the lineage added a capability without giving up speed: Mercury 2 introduced reasoning to diffusion models, Mercury 2.5 added tunable reasoning and a 260K context window, and Mercury 3 delivers the largest intelligence gain of the series plus a context window expanded to 1 million tokens.

The pricing is equally aggressive: $0.20 per million input and $0.70 per million output tokens, halved to $0.10/$0.35 at launch, with cached tokens 90% off. And because the API is OpenAI-compatible and available through the Inception API, OpenRouter, Baseten and Vercel from day one, teams can switch over without rewriting code.

What Actually Is a Diffusion LLM?

Nearly every chatbot you've used — ChatGPT, Claude, Gemini — is an autoregressive model: it writes strictly left to right, one token at a time, each word conditioned on all the words before it. It's a clean, proven design with one built-in cost: you cannot start the next token until the current one is done. Latency scales with answer length.

Diffusion models come from the image-generation world (Stable Diffusion, Midjourney). Instead of writing token by token, they start with a block of noise and refine the entire answer in parallel over a handful of denoising steps. Because many tokens get settled simultaneously in each pass, generation isn't chained to sequence length — which is how Mercury sustains four-digit token counts per second. A useful side effect: diffusion models can revise earlier parts of an answer mid-generation, rather than committing to a flawed opening and defending it to the end.

The catch, historically, was capability: early dLLMs were fast but thin. Mercury 3 is the strongest public evidence yet that diffusion architectures can scale up the intelligence curve — Inception calls it a new Pareto frontier in speed and intelligence, and at minimum it's now competing with the small frontier tier, not the toy tier. Early customers are already running it in browser-use agents, interactive education, real-time search agents, and customer support — the four product categories where waiting two seconds per paragraph is disqualifying.

Why 1,000 Tokens a Second Changes Products

Speed at this level isn't a benchmark brag; it moves product categories from "demo" to "shippable":

• Agents stop feeling sluggish. A browser or search agent that needs to produce a plan, read a page, and write a synthesis does 5–20 model calls per task. At 50 tok/s that's tens of seconds of dead air; at 1,000 tok/s the loop feels conversational. This is why agent-first customers are Inception's earliest adopters.
• Latency budget shifts to the model. When generation is nearly free time-wise, teams stop truncating reasoning and stop choosing dumber models to save seconds — you can afford the full-strength model on every keystroke-level interaction.
• Whole-document context becomes interactive. A 1M-token window plus instant output means you can genuinely converse with a large codebase, contract set, or research corpus instead of chunking and praying.
• Voice and live UI unlock. Spoken responses and streaming interfaces need tokens faster than humans read (~5–8 tok/s) plus headroom for reasoning. Sub-second first-token experiences are now an API key away — try tools like Claude Haiku 5.5 for the incumbent take on fast and cheap, then benchmark Mercury against it yourself.

The Same-Day Context: a Brutal Price-and-Speed War

Mercury 3 didn't land in a vacuum — October 19 was a two-front assault on the cost of intelligence. China's MiniMax launched M3.1 Flash the same morning, pricing input at $0.15 per million tokens and output at $0.60 — a 70% cut from its predecessor — with weights promised for October 23. MiniMax posts strong agent scores for it too (73.7% on DeepSWE v1.1, 61.7 on OSWorld V2), and confirmed that the stealth "Space Bunny" model that briefly surfaced on OpenRouter in late September was an earlier build.

Stack that on last week's DeepSeek V4.1-Flash-1017 — a 552B-parameter open-weight MoE (MIT license) with 1M-token context that edges past Claude Opus 5 on several coding-agent benchmarks at a fraction of the price — and the pattern is unmistakable: the "good enough frontier" is collapsing in price by 50–70% per quarter, from at least three independent directions: new architectures (diffusion), new training economics (MiniMax), and ruthless efficiency engineering (DeepSeek). If your AI bill hasn't dropped this year, your stack is stale — compare current options on DeepSeek and Baseten against whatever you locked in last winter.

What This Means for the AI Tools You Choose

The two-year question isn't whether diffusion LLMs replace autoregressive ones — it's whether "wait for the model" remains an acceptable product experience at all. Today, the answer started tilting.

Frequently Asked Questions