Mercury: Ultra-Fast Language Models Based on Diffusion

Samar Khanna

Abstract:

We present Mercury, a new generation of commercial-scale large language models (LLMs)
based on diffusion. These models are parameterized via the Transformer architecture and
trained to predict multiple tokens in parallel. In this report, we detail Mercury Coder, our first
set of diffusion LLMs designed for coding applications. Currently, Mercury Coder comes in
two sizes: Mini and Small. These models set a new state-of-the-art on the speed-quality fron-
tier. Based on independent evaluations conducted by Artificial Analysis, Mercury Coder Mini
and Mercury Coder Small achieve state-of-the-art throughputs of 1109 tokens/sec and 737 to-
kens/sec, respectively, on NVIDIA H100 GPUs and outperform speed-optimized frontier models
by up to 10×on average while maintaining comparable quality. We discuss additional results
on a variety of code benchmarks spanning multiple languages and use-cases as well as real-world
validation by developers on Copilot Arena, where the model currently ranks second on quality
and is the fastest model overall. We also release a public API at platform.inceptionlabs.ai
and free playground at chat.inceptionlabs.ai.