Improved Denoising Diffusion Probabilistic Models

Alexander Quinn Nichol, Prafulla Dhariwal

Keywords: neural networks, generative models, log-likelihood, diffusion models, denoising diffusion probabilistic models, image generation

Abstract: We explore denoising diffusion probabilistic models, a class of generative models which have recently been shown to produce excellent samples in the image and audio domains. While these models produce excellent samples, it has yet to be shown that they can achieve competitive log-likelihoods. We show that, with several small modifications, diffusion models can achieve competitive log-likelihoods in the image domain while maintaining high sample quality. Additionally, our models allow for sampling with an order of magnitude fewer diffusion steps with only a modest difference in sample quality. Finally, we explore how sample quality and log-likelihood scale with the number of diffusion steps and the amount of model capacity. We conclude that denoising diffusion probabilistic models are a promising class of generative models with excellent scaling properties and sample quality.

One-sentence Summary: We show that denoising diffusion probabilistic models can achieve competitive log-likelihoods and efficient sampling.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2102.09672/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=qoEfaqmopt