Expressiveness Remarks for Denoising Diffusion Based Sampling

Francisco Vargas, Teodora Reu, Anna Kerekes

Keywords: DDPM, Diffusion Probabilistic Models, Score Matching, Stochastic Control, Foellmer Drift, Schroedinger Bridges

TL;DR: We prove universal approximation results with minimal assumptions for DDPM in the context of sampling via extending previous approximation results for the Foellmer drift.

Abstract: Denoising diffusion models are a class of generative models which have recently achieved state-of-the-art results across many domains. Gradual noise is added to the data using a diffusion process, which transforms the data distribution into a Gaussian. Samples from the generative model are then obtained by simulating an approximation of the time reversal of this diffusion initialized by Gaussian samples. Recent research has explored adapting diffusion models for sampling and inference tasks. In this paper, we leverage known connections to stochastic control akin to the F\"ollmer drift to extend established neural network approximation results for the F\"ollmer drift to denoising diffusion models and samplers.