DDPM Score Matching Is Asymptotically Efficient

Sinho Chewi, Alkis Kalavasis, Anay Mehrotra, Omar Montasser

Track: tiny / short paper (up to 4 pages)

Keywords: Score Matching, DDPM, Asymptotic Efficiency, Diffusion

TL;DR: We show that DDPM Score Matching gives rise to asymptotically efficient and normal estimators illustrating their statistical power; and demonstrating stark contrast to other score matching variants (e.g., implicit score matching).

Abstract: The success of score-based generative models (SGMs), and particularly denoising diffusion probabilistic models (DDPMs), rests on the statistical technique of *score matching*, for which rigorous guarantees are nascent. In fact, recent work has shown that for estimation in parametric models, a variant of score matching known as implicit score matching is provably statistically inefficient for multimodal densities that are common in practice. In contrast, under mild conditions, we show that denoising score matching in DDPMs is asymptotically efficient, i.e., the DDPM estimator is asymptotically normal with covariance matrix given by the inverse Fisher information. Our proof is based on a pointwise relationship between the empirical risks of DDPM and maximum likelihood estimation.

Submission Number: 106