Enhancing Sample Generation of Diffusion Models using Noise Level Correction
Abstract: The denoising process of diffusion models can be interpreted as an approximate projection of noisy samples onto the data manifold. Moreover, the noise level in these samples approximates their distance to the underlying manifold. Building on this insight, we propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold. Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process. Additionally, we extend this approach to various image restoration tasks by integrating task-specific constraints, including inpainting, deblurring, super-resolution, colorization, and compressed sensing. Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios. Notably, the proposed noise level correction framework is compatible with existing denoising schedulers (e.g., DDIM), offering additional performance improvements.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We have revised the paper based on the reviewer’s comments. Major modifications are highlighted in magenta. Key changes include:
1. Added detailed discussion of the scaling factor $\lambda$ in training and included further explanation in Appendix D.3.
2. Rewrote the Related Work section to incorporate additional relevant studies.
3. Extended the proposed NLC method to DPM-Solver, with implementation details in Appendix B and corresponding experiments in Appendix D.5.
4. Provided a comparison of time complexity and runtime overhead across methods in Appendix D.2.
5. Added more explanation and derivation of the training objective in Appendix A.2.
6. Fixed various typos throughout the paper.
Assigned Action Editor: ~Shiyu_Chang2
Submission Number: 3933
Loading