The Accumulation of Score Estimation Error in Diffusion Models

Baoxiang He, Valentio Iverson, Shuai Li, Cheng Chen, Bo Jiang

Keywords: Diffusion models, Score estimation error

TL;DR: We analyze how score estimation error accumulates in diffusion models and provide Wasserstein bounds between the final and target distributions, explaining empirical differences across discretization schedules and samplers.

Abstract: Diffusion models are widely used for high-quality generation, but their performance is sensitive to the accuracy of the estimated score. Our main results are established first in the setting where the forward process is initialized from a Gaussian mixture, where we derive Wasserstein bounds by leveraging the structure of the score and its Hessian. We then extend the analysis to general data distributions, where we provide a more general but looser upper bound. Our analysis reveals how discretization steps directly shape the accumulation of score estimation error, thereby explaining previously observed empirical phenomena regarding the advantage of certain discretization schedules. In addition, we show that, in the Gaussian setting, SDE samplers accumulate less error than ODE samplers in the small step-size regime, which explains their superior empirical performance. The result holds for both variance-preserving (VP) and variance-exploding (VE) diffusions.

Primary Area: generative models

Submission Number: 10288