An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models

Binxu Wang, Cengiz Pehlevan

Keywords: Spectral bias, analytical theory, learning dynamics, diffusion, flow matching, deep linear network

TL;DR: Closed‑form analysis of diffusion learning dynamics uncovers an inverse‑variance law of distributional convergence; MLP‑UNets comply, while convolutional U‑Nets break it.

Abstract:

We develop an analytical framework for understanding how the learned distribution evolves during diffusion model training.
Leveraging the Gaussian equivalence principle, we derived exact solutions for the gradient-flow dynamics of weights in one or two layer linear or linear convolutional denoiser settings with arbitrary data, where linear networks converge along principal components, and convolutional networks converge along Fourier modes.
Remarkably, these solutions allow us to derive the generated distribution in closed-form and its KL-divergence through training.
These analytical results expose a pronounced \emph{spectral bias}, i.e. for both weights and generated distributions, the convergence time of a mode follows an inverse power law of its variance.
Empirical experiments on both Gaussian and natural image datasets demonstrate that the power-law spectral bias—remain robust even when using deeper or convolutional architectures.
Our results underscore the importance of the data covariance in dictating the order and rate at which diffusion models learn different modes of the data, providing potential explanations of why earlier stopping could lead to incorrect details in image generative model.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 23092