Enhancing Sample Generation of Diffusion Models using Noise Level Correction

Abulikemu Abuduweili, Chenyang Yuan, Changliu Liu, Frank Permenter

Abstract: The denoising process of diffusion models can be interpreted as an approximate projection of noisy samples onto the data manifold. Moreover, the noise level in these samples approximates their distance to the underlying manifold. Building on this insight, we propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold. Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process. Additionally, we extend this approach to various image restoration tasks by integrating task-specific constraints, including inpainting, deblurring, super-resolution, colorization, and compressed sensing. Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios. Notably, the proposed noise level correction framework is compatible with existing denoising schedulers (e.g., DDIM), offering additional performance improvements.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

1. The code has been open-sourced and is available at: https://github.com/Walleclipse/Diffusion-NLC/

2. The paper has been polished.

Code: https://github.com/Walleclipse/Diffusion-NLC/

Assigned Action Editor: ~Shiyu_Chang2

Submission Number: 3933