Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty

ICLR 2026 Conference Submission19972 Authors

Keywords: Masked Diffusion Models, Sampling, Steering, Uncertainty, Inference-Time Strategy

TL;DR: We guide the generation of Masked Diffusion Models using a new Denoising Entropy metric, turning uncertainty into an advantage to significantly boost output quality.

Abstract: Masked Diffusion Models (MDMs) offer flexible, non-autoregressive generation, but this freedom introduces a challenge: final output quality is highly sensitive to the decoding order. We are the first to formalize this issue, attributing the variability in output quality to the cumulative predictive uncertainty along a generative path. To quantify this uncertainty, we introduce Denoising Entropy, a computable metric that serves as an internal signal for evaluating generative process. Leveraging this metric, we propose two algorithms designed to optimize the decoding path: a post-hoc selection method and a real-time guidance strategy. Experiments demonstrate that our entropy-guided methods significantly improve generation quality, substantially boosting accuracy on challenging reasoning, planning, and code benchmarks. Our work establishes Denoising Entropy as a principled tool for understanding and controlling generation, effectively turning the uncertainty in MDMs from a liability into a key advantage for discovering high-quality solutions.

Primary Area: generative models

Submission Number: 19972