Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models

TMLR Paper9538 Authors

Abstract: Diffusion generative models synthesize samples by discretizing reverse-time dynamics driven by a learned score or denoiser. Existing convergence analyses often exhibit explicit dependence on the ambient dimension, while dimension-free guarantees typically require structural or geometric assumptions on the target distribution. We develop an information-theoretic approach to reverse-diffusion discretization that avoids such assumptions. We decompose the pathwise KL error into initialization, denoiser approximation and time-discretization terms, and express the discretization term exactly through the MMSE curve of the associated Gaussian channel. Under finite second moment and finite R\'enyi entropy of order $1/2$, we obtain a dimension-free discretization bound controlled by the R\'enyi entropy and the number of sampling steps. Motivated by the same decomposition, we propose a Loss-Adaptive Schedule (LAS), an algorithmic scheduling rule that uses training-loss information to allocate sampling steps across noise levels. Experiments show that LAS improves sampling quality over standard heuristic schedules, especially in low-step regimes.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Kangwook_Lee1

Submission Number: 9538