Solving Inverse Problems using Diffusion with Iterative Colored Renoising

TMLR Paper4646 Authors

Abstract:

Imaging inverse problems can be solved in an unsupervised manner using pre-trained diffusion models, but doing so requires approximating the gradient of the measurement-conditional score function in the diffusion reverse process.
We show that the approximations produced by existing methods are relatively poor, especially early in the reverse process, and so
we propose a new approach that iteratively reestimates and ``renoises'' the estimate several times per diffusion step.
This iterative approach, which we call Fast Iterative REnoising (FIRE), injects colored noise that is shaped to ensure that the pre-trained diffusion model always sees white noise, in accordance with how it was trained.
We then embed FIRE into the DDIM reverse process and show that the resulting ``DDfire'' offers state-of-the-art accuracy and runtime on several linear inverse problems, as well as phase retrieval.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

The main changes since the last submission are the following:
* Clarifications to the Introduction
* Clarifications to the Background section, including a discussion of CSGM [Jalal'21], PnP-ULA [Laumont'22], GPnP [Bouman'23], SNORE [Renaud'24]
* Rewriting Section 3.1 on SLM-FIRE for improved readability
* Rewriting Section 3.4 on relation to other methods for clarity
* Numerical comparison to DDS
* A new Appendix I that includes DDfire hyperparameter tuning curves
* A new Appendix G that includes detailed background on DDS, DiffPIR, and SNORE, as well as detailed comparisons to FIRE/DDfire.
* Phase-retrieval experiments now include ImageNet data as well as FFHQ
* An extension of DDfire that takes guidance from an initialization is described in the Appendix

Assigned Action Editor: ~Qing_Qu2

Submission Number: 4646