Diffusion Models are Minimax Optimal Distribution Estimators

Kazusato Oko, Shunta Akiyama, Taiji Suzuki

Keywords: diffusion models, score-based generative models, score estimation, distribution estimation, approximation theory

TL;DR: Analyzed diffusion modeling and theoretically proved that the distribution generated by diffusion modeling achieves the minimax optimal estimation rates in the total variation distance and in the Wasserstein distance of order one.

Abstract: This paper provides the first rigorous analysis of estimation error bounds of diffusion modeling, trained with a finite sample, for well-known function spaces. The highlight of this paper is that when the true density function belongs to the Besov space and the empirical score matching loss is properly minimized, the generated data distribution achieves the nearly minimax optimal estimation rates in the total variation distance and in the Wasserstein distance of order one. Furthermore, we extend our theory to demonstrate how diffusion models adapt to low-dimensional data distributions. We expect these results advance theoretical understandings of diffusion modeling and its ability to generate verisimilar outputs.