On the Complexity Theory of Masked Discrete Diffusion: From $\mathrm{poly}(1/\epsilon)$ to Nearly $\epsilon$-Free

ICLR 2026 Conference Submission16930 Authors

Keywords: masked discrete diffusion, inference analysis, uniformization, epsilon free complexity

Abstract:

We study *masked discrete diffusion*---a flexible paradigm for text generation in which
tokens are progressively corrupted by special mask symbols before being denoised.
Although this approach has demonstrated strong empirical performance, its theoretical complexity in high-dimensional settings remains insufficiently understood.
Existing analyses largely focus on *uniform* discrete diffusion, and more recent attempts addressing masked diffusion either (1) overlook widely used Euler samplers, (2) impose restrictive bounded-score assumptions, or (3) fail to showcase the advantages of masked discrete diffusion over its uniform counterpart.
To address this gap, we show that Euler samplers can achieve \(\epsilon\)-accuracy in total variation (TV) with $\tilde{O}(d^{2}\epsilon^{-3/2})$ discrete score evaluations, thereby providing the first rigorous analysis of typical Euler sampler in masked discrete diffusion.
We then propose a *Mask-Aware Truncated Uniformization* (MATU) approach that both removes bounded-score assumptions and preserves unbiased discrete score approximation.
By exploiting the property that each token can be unmasked at most once, MATU attains a nearly $\epsilon$-free complexity of $O(d\ln d\cdot (1-\epsilon^2))$.
This result surpasses existing uniformization
methods under uniform discrete diffusion, eliminating the $\ln(1/\epsilon)$ factor and substantially speeding up convergence.
Our findings not only provide a rigorous theoretical foundation for masked discrete diffusion, showcasing its practical advantages over uniform diffusion for text generation, but also
pave the way for future efforts to analyze diffusion-based language models developed under masking paradigm.

Primary Area: learning theory

Submission Number: 16930