Constructing Non-isotropic Gaussian Diffusion Model Using Isotropic Gaussian Diffusion Model for Image Editing

Xi Yu, Xiang Gu, Haozhi Liu, Jian Sun

Keywords: score-based diffusion model, non-isotropic Gaussian diffusion model, image editing

TL;DR: We propose a non-isotropic Gaussian diffusion model (NGDM) for image editing. By rectifying the NGDM, we can generate image by NGDM based on the pre-trained isotropic Gaussian diffusion model, e.g., DDPM.

Abstract: Score-based diffusion models (SBDMs) have achieved state-of-the-art results in image generation. In this paper, we propose a Non-isotropic Gaussian Diffusion Model (NGDM) for image editing, which requires editing the source image while preserving the image regions irrelevant to the editing task. We construct NGDM by adding independent Gaussian noises with different variances to different image pixels. Instead of specifically training the NGDM, we rectify the NGDM into an isotropic Gaussian diffusion model with different pixels having different total forward diffusion time. We propose to reverse the diffusion by designing a sampling method that starts at different time for different pixels for denoising to generate images using the pre-trained isotropic Gaussian diffusion model. Experimental results show that NGDM achieves state-of-the-art performance for image editing tasks, considering the trade-off between the fidelity to the source image and alignment with the desired editing target.

Submission Number: 6113