Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration

Published: 27 Jun 2023, Last Modified: 02 Apr 2024Accepted by TMLREveryoneRevisionsBibTeX
Event Certifications:
Abstract: Inversion by Direct Iteration (InDI) is a new formulation for supervised image restoration that avoids the so-called ``regression to the mean'' effect and produces more realistic and detailed images than existing regression-based methods. It does this by gradually improving image quality in small steps, similar to generative denoising diffusion models. Image restoration is an ill-posed problem where multiple high-quality images are plausible reconstructions of a given low-quality input. Therefore, the outcome of a single step regression model is typically an aggregate of all possible explanations, therefore lacking details and realism. The main advantage of InDI is that it does not try to predict the clean target image in a single step but instead gradually improves the image in small steps, resulting in better perceptual quality. While generative denoising diffusion models also work in small steps, our formulation is distinct in that it does not require knowledge of any analytic form of the degradation process. Instead, we directly learn an iterative restoration process from low-quality and high-quality paired examples. InDI can be applied to virtually any image degradation, given paired training data. In conditional denoising diffusion image restoration the denoising network generates the restored image by repeatedly denoising an initial image of pure noise, conditioned on the degraded input. Contrary to conditional denoising formulations, InDI directly proceeds by iteratively restoring the input low-quality image, producing high-quality results on a variety of image restoration tasks, including motion and out-of-focus deblurring, super-resolution, compression artifact removal, and denoising.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: In the camera ready version we incorporate the changes mentioned in the discussion with the reviewers: 1. Fixed typos and uniformize notiation, and the presentation of results in tables (colorized best and second best method) 2. Added missing references and other concurrent work that appear in the last couple of months in ArXiv. 3. Added Appendix B -- Denoising with a Gaussian Prior. 4. Added a short discussion on InDI Residual Flow as requested by the reviewers (page 6).
Certifications: Featured Certification
Assigned Action Editor: ~Jia-Bin_Huang1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 978