Edge-preserving noise for diffusion models

Jente Vandersanden, Sascha Holl, Xingchang Huang, Gurprit Singh

Keywords: diffusion, generative modelling, denoising

TL;DR: We introduce a content-aware diffusion model that is explicitly trained to learn the non-isotropic edge information in a dataset.

Abstract:

Classical generative diffusion models learn an isotropic Gaussian denoising process,
treating all spatial regions uniformly, thus neglecting potentially valuable structural
information in the data. Inspired by the long-established work on anisotropic
diffusion in image processing, we present a novel edge-preserving diffusion model
that is a generalization of denoising diffusion probablistic models (DDPM). In
particular, we introduce an edge-aware noise scheduler that varies between edgepreserving
and isotropic Gaussian noise. We show that our model’s generative
process converges faster to results that more closely match the target distribution.
We demonstrate its capability to better learn the low-to-mid frequencies within the
dataset, which plays a crucial role in representing shapes and structural information.
Our edge-preserving diffusion process consistently outperforms state-of-the-art
baselines in unconditional image generation. It is also more robust for generative
tasks guided by a shape-based prior, such as stroke-to-image generation. We
present qualitative and quantitative results showing consistent improvements (FID
score) of up to 30% for both tasks.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10879