Denoising Diffusion Implicit Models

Jiaming Song, Chenlin Meng, Stefano Ermon

Keywords: generative models, variational autoencoders, denoising score matching, variational inference

Abstract: Denoising diffusion probabilistic models (DDPMs) have achieved high quality image generation without adversarial training, yet they require simulating a Markov chain for many steps in order to produce a sample. To accelerate sampling, we present denoising diffusion implicit models (DDIMs), a more efficient class of iterative implicit probabilistic models with the same training procedure as DDPMs. In DDPMs, the generative process is defined as the reverse of a particular Markovian diffusion process. We generalize DDPMs via a class of non-Markovian diffusion processes that lead to the same training objective. These non-Markovian processes can correspond to generative processes that are deterministic, giving rise to implicit models that produce high quality samples much faster. We empirically demonstrate that DDIMs can produce high quality samples $10 \times$ to $50 \times$ faster in terms of wall-clock time compared to DDPMs, allow us to trade off computation for sample quality, perform semantically meaningful image interpolation directly in the latent space, and reconstruct observations with very low error.

One-sentence Summary: We show and justify a GAN-like iterative generative model with relatively fast sampling, high sample quality and without any adversarial training.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ ermongroup/ddim](https://github.com/ermongroup/ddim) + [ 23 community implementations](https://paperswithcode.com/paper/?openreview=St1giarCHLP)

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:2010.02502/code)