Restorer Guided Diffusion Models for Variational Inverse Problems

JingHao Zhang, Zizheng Yang, Man Zhou, Feng Zhao

Keywords: Diffusion model, posterior sampling, restorer guidance

Abstract: Diffusion models have made remarkable progress in solving various inverse problems, attributing to the generative modeling capability of the data manifold. Posterior sampling from the conditional score function enable the precious data consistency powered by the measurement-based likelihood term. However, most prevailing approaches confined to the insufficient expressive ability of the measurement model with merely digitized measuring deterioration, regardless of complicated unpredictable disturbance in real-world sceneries. To address this, we show that the measurement-based likelihood can be renewed with restoration-based likelihood, licencing the patronage of various off-the-shelf restoration models for powerful diffusion solvers, in what we call restorer guidance. Particularly, assembled with versatile restorer guidance optionally, we can resolve inverse problems with bunch of choices for assorted sample quality and realize the proficient deterioration control with assured realistic. We show that our work can be analogous to the transition from the classifier guidance to classifier-free guidance in the field of inverse problem solver. Experiments on various complicated inverse problems illustrate the effectiveness of our method, including image dehazing, rain streak removal, and motion deblurring. Code will be available soon.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7138