Inception Mercury 3: how a diffusion LLM hits 1,000 tokens/s

Mercury 3 is a diffusion language model from Inception, released on October 19, 2026. It doesn't write text one token at a time. It drafts the whole response and then refines all of it in parallel, and Inception measures it at 1,068 output tokens per second on NVIDIA GPUs. It is available through an OpenAI-compatible API with a 1M-token context window, tool calling and structured outputs.

So the speed comes from a different way of generating text, not from faster hardware. That changes a few practical things for you: how latency behaves, which settings matter, and which workloads get the most out of it. Here is how it works and how to call it.

What is Mercury 3?

Mercury 3 is the newest model in Inception's Mercury family and the largest so far. Inception says it is more than 40 times larger than Mercury 1, though it hasn't published a parameter count. Mercury 2 came out in February 2026 and Mercury 2.5 in September, so this is the third release this year.

It is offered as a hosted model only. Inception hasn't announced open weights. The published specs:

It runs on Inception's own API platform and through OpenRouter, Baseten and Vercel's AI Gateway. New platform accounts get 100 million free tokens.

How does a diffusion language model work?

Most LLMs are autoregressive: each forward pass through the network produces exactly one new token, which is appended to the text and fed back in for the next pass. A 1,000-token answer needs 1,000 sequential passes. You can make each pass faster, but you can't skip the chain, because token 500 depends on token 499.

A diffusion model turns this around. The idea comes from image generators, which start from random noise and remove it step by step until a picture shows up. For text, the model starts with a sequence of placeholder positions, a rough draft of the whole answer, and on each step it predicts better tokens for every position at once. After a few refinement passes, the draft settles into the final text.

The key difference is what one forward pass buys you:

• Autoregressive: one pass, one new token. Output length sets the number of passes.
• Diffusion: one pass, an update to many positions in parallel. The number of refinement steps sets the cost, and it can be much smaller than the number of tokens.

GPUs are good at doing a lot of work in parallel and comparatively bad at long chains of small sequential steps. Autoregressive decoding is mostly that second kind of work, so a lot of the GPU sits idle waiting on memory. Diffusion gives the hardware bigger chunks of parallel work per step, which is where the throughput comes from.

There's a second effect. Because the model revises earlier positions, it can fix a mistake in the draft before you see it. An autoregressive model has already committed to every token it has emitted.

Inception hasn't published how many refinement steps Mercury 3 uses or how its decoding schedule works. What it has said is that the scaling behavior known from autoregressive models also holds for diffusion: bigger models trained on more data get better in a predictable way. Mercury 3 is its argument for that claim.

How do you call the Mercury 3 API?

The API follows the OpenAI chat completions format, so existing client code mostly just needs a new base URL and model name. Here is the request from Inception's docs:

export INCEPTION_API_KEY="your_api_key_here"

curl https://api.inceptionlabs.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $INCEPTION_API_KEY" \
  -d '{
    "model": "mercury-3",
    "messages": [
      {"role": "user", "content": "What is a diffusion model?"}
    ],
    "reasoning_effort": "medium",
    "max_completion_tokens": 8192
  }'

Inception also ships its own SDKs, inceptionai for Python and TypeScript, with the same method shape:

from inceptionai import Inception

client = Inception()  # reads INCEPTION_API_KEY from the environment

completion = client.chat.completions.create(
    model="mercury-3",
    messages=[{"role": "user", "content": "What is a diffusion model?"}],
    reasoning_effort="medium",
    max_completion_tokens=8192,
)
print(completion.choices[0].message.content)

Two details are easy to miss. First, the default reasoning effort for mercury-3 is low, while Mercury 2 and 2.5 default to medium. The doc samples set medium explicitly, so if you copy a sample and then remove the parameter, you get different behavior. Second, temperature only goes from 0.5 to 1.0. If your existing code sends temperature: 0 for deterministic output, check how the API handles it before you depend on it.

Which reasoning effort should you pick?

The effort setting controls how much thinking the model does before it answers. Inception describes the four levels roughly like this:

• instant: voice, real-time turns and high-volume automation
• low: latency-sensitive chat and tool calling (the default)
• medium: general-purpose use
• high: complex, multi-step reasoning

Higher effort means more generated tokens and more time, the same trade-off you get with any reasoning model. The difference is that each extra token costs less wall-clock time here, so you can use more reasoning before latency becomes the bottleneck.

When should you use Mercury 3?

Speed pays off most where a person or a program is waiting on many model calls in a row:

• Agent loops. An agent might call the model dozens of times per task: plan, call a tool, read the result, decide again. Per-call latency adds up across the whole task, so cutting it shortens every step.
• Voice and real-time interfaces. The instant level is meant for turn-taking where a pause of a second or two feels broken.
• Interactive coding help. Inline edits, hints and quick rewrites, where you want the answer before you lose your train of thought.
• Bulk processing. Classification, extraction and summarization over large batches, where throughput sets your total job time.

If you've looked at paying more for a faster tier of an autoregressive model, like OpenAI's Ultrafast service tier, this is a different route to the same goal: change how the model generates instead of buying priority on the hardware.

There are also reasons to hold off. Inception's benchmark numbers come from its own evaluations, so run your own prompts before you switch a production workload. The model is text only, so image or audio input needs something else. And since the weights aren't public, you can't self-host it today.

A sensible way to try it: send a slice of real traffic to mercury-3 behind the same OpenAI-compatible client, log latency and output quality next to your current model, and compare. Because the request shape is the same, switching back is a config change.

Key takeaways

• Mercury 3 is a diffusion LLM: it refines a full draft in parallel instead of emitting one token per forward pass.
• Inception measures it at 1,068 tokens per second, with a 1M-token context and up to 131,072 output tokens.
• The API is OpenAI-compatible, uses model ID mercury-3, and supports tool calling and structured outputs.
• Default reasoning effort is low, and temperature is limited to 0.5 to 1.0. Both can surprise ported code.
• It fits best in agent loops, voice and other latency-bound workloads. Test it on your own prompts first.

FAQ

Is Mercury 3 open source?

No. Mercury 3 is available only through Inception's API and partner platforms such as OpenRouter, Baseten and Vercel. Inception hasn't announced open weights.

Can I use the OpenAI SDK with Mercury 3?

Inception describes the API as OpenAI-compatible, using the standard /v1/chat/completions endpoint at https://api.inceptionlabs.ai/v1. Pointing an OpenAI-style client at that base URL with model mercury-3 should work for chat requests. Inception's own inceptionai SDK is the documented option.

Why is a diffusion LLM faster than a regular LLM?

A regular LLM needs one sequential forward pass per output token. A diffusion LLM updates many positions in each pass and needs only a few refinement passes, which keeps the GPU busy with parallel work instead of waiting between tiny steps.