Wasserstein Bounds for generative diffusion models with Gaussian tail targets

TMLR Paper6742 Authors

Abstract:

We present an estimate of the Wasserstein distance between the data distribution and the generation of score-based generative models. The sampling complexity with respect to dimension is $\mathcal{O}(\sqrt{d})$, with a logarithmic constant. In the analysis, we assume a Gaussian-type tail behavior of the data distribution and an $\epsilon$-accurate approximation of the score. Such a Gaussian tail assumption is general, as it accommodates practical target distributions derived from early stopping techniques with bounded support.

The crux of the analysis lies in the global Lipschitz bound of the score, which is shown from the Gaussian tail assumption by a dimension-independent estimate of the heat kernel. Consequently, our complexity bound scales linearly (up to a logarithmic constant) with the square root of the trace of the covariance operator, which relates to the invariant distribution of the forward process.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=KRRzRNgNp9

Changes Since Last Submission:

1. The second paragraph of the Introduction has been revised. It now explicitly introduces the two major "camps" regarding score function regularity (Lipschitz bound available vs. unavailable) and includes a number of new citations to works that represent progress in this direction.

2. The Preliminaries section was reorganized to improve the logical flow between continuous-time and discrete-time settings. This involved creating explicit, sequentially ordered subsections for:
- General Notations，
- Continuous-time formulation of diffusion models，
- Training and discrete-time approximation of diffusion models，
- Heat Kernel Estimation.
3. The introduction to Section 3 (Results) has been improved to more relevant to the ML readership. Connecting text is add to help readers transition from one statement to the next.

4. A new Section 4 ( titled "Conclusion" ) was added to provide a recap of the paper's main contributions and propose directions for future research.

Assigned Action Editor: ~Omar_Rivasplata1

Submission Number: 6742