Soft Diffusion: Score Matching For General Corruptions

Giannis Daras, Mauricio Delbracio, Hossein Talebi, Alex Dimakis, Peyman Milanfar

Keywords: diffusion, score-based models, generative models

TL;DR: We define a broader family of corruption processes that generalizes previously known diffusion models and we show how to learn to reverse them.

Abstract:

We define a broader family of corruption processes that generalizes previously known diffusion models. To reverse these general diffusions, we propose a new objective called Soft Score Matching that provably learns the score function for any linear corruption process and yields state of the art results for CelebA. Soft Score Matching incorporates the degradation process in the network.
Our new loss trains the model to predict a clean image, that after corruption, matches the diffused observation.
We show that our objective learns the gradient of the likelihood under suitable regularity conditions for a family of corruption processes.
We further develop a principled way to select the corruption levels for general diffusion processes and a novel sampling method that we call Momentum Sampler.
We show experimentally that our framework works for general linear corruption processes, such as Gaussian blur and masking.
We achieve state-of-the-art FID score $1.85$ on CelebA-64, outperforming all previous linear diffusion models.
We also show significant computational benefits compared to vanilla denoising diffusion.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Generative models