Deterministic Discrete Denoising

ICLR 2026 Conference Submission8455 Authors

Keywords: discrete diffusion, deterministic denoising, herding algorithm, discrete generative modeling

TL;DR: Derandomization of the generative denoising process for discrete-state diffusion models

Abstract:

We propose a deterministic denoising algorithm for discrete-state diffusion models based on Markov chains.
The generative reverse process is derandomized by introducing a variant of the herding algorithm
with weakly chaotic dynamics, which induces deterministic discrete state transitions.
Our approach is a direct replacement for the stochastic denoising process,
requiring neither retraining nor continuous state embeddings.
We demonstrate consistent improvements in both efficiency and sample quality on text and image generation tasks.
Thus, this simple derandomization approach is expected to enhance the significance of discrete diffusion in generative modeling.
Furthermore, our results reveal that deterministic reverse processes, well established in continuous diffusion,
can also be effective in discrete state spaces.

Primary Area: generative models

Submission Number: 8455