Score-Based Denoising Diffusion Models for Photon-Starved Image Restoration Problems

Savvas Melidonis, Yiming Xi, Konstantinos C. Zygalakis, Yoann Altmann, Marcelo Pereyra

Abstract: Score-based denoising diffusion models have recently emerged as a powerful strategy to solve image restoration problems. Early diffusion models required problem-specific training. However, modern approaches can combine a likelihood function that is specified during test-time with a foundational pretrained diffusion model, which is used as an implicit prior in a Plug-and-Play (PnP) manner. This approach has been shown to deliver state-of-the-art performance in a wide range of image restoration problems involving Gaussian and mild Poisson noise. With extreme computer vision applications in mind, this paper presents the first PnP denoising diffusion method for photon-starved imaging problems. These problems arise in new quantum-enhanced imaging systems that exploit the particle nature of light to exceed the limitations of classical imaging. The problems involve highly challenging noise statistics, such as binomial, geometric, and low-intensity Poisson noise, which are difficult because of high uncertainty about the solution and because the models exhibit poor regularity properties (e.g., exploding scores, constraints). The proposed method is demonstrated on a series of challenging photon-starved imaging experiments with as little as 1 photon per pixel, where it delivers remarkably accurate solutions and outperforms alternative strategies from the state-of-the-art.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: We sincerely thank all the reviewers for their valuable and insightful feedback, as well as the editor for their time and effort in handling our submission. In the camera-ready version, we have revised the introduction to better present the research problem and emphasize our improvements over existing methods. We have also added a table comparing the computational time of diffusion models to provide a clearer analysis of efficiency, included the missing citations, expanded the discussion especially in comparing LPIPS, and refined mathematical expressions to correct minor issues. We appreciate the reviewers' thoughtful comments, which have helped us enhance the clarity and quality of our work.

Code: https://github.com/SavvasMel/Prox-DiffPIR

Assigned Action Editor: ~Jeremias_Sulam1

Submission Number: 3339