Stochastic Deep Restoration Priors for Imaging Inverse Problems

Yuyang Hu, Albert Peng, Weijie Gan, Peyman Milanfar, Mauricio Delbracio, Ulugbek S. Kamilov

TL;DR: We introduce ShaRP, a novel framework that stochastically leverages an ensemble of deep restoration models beyond denoisers to regularize inverse problems.

Abstract: Deep neural networks trained as image denoisers are widely used as priors for solving imaging inverse problems. We introduce Stochastic deep Restoration Priors (ShaRP), a novel framework that stochastically leverages an ensemble of deep restoration models beyond denoisers to regularize inverse problems. By using generalized restoration models trained on a broad range of degradations beyond simple Gaussian noise, ShaRP effectively addresses structured artifacts and enables self-supervised training without fully sampled data. We prove that ShaRP minimizes an objective function involving a regularizer derived from the score functions of minimum mean square error (MMSE) restoration operators. We also provide theoretical guarantees for learning restoration operators from incomplete measurements. ShaRP achieves state-of-the-art performance on tasks such as magnetic resonance imaging reconstruction and single-image super-resolution, surpassing both denoiser- and diffusion-model-based methods without requiring retraining.

Lay Summary:

Recovering clear images from noisy or incomplete data, like MRIs or low-resolution photos, is a common challenge. While deep learning models often help by learning to remove simple noise, this isn't always enough for complex errors.

We introduce "ShaRP," a new system that uses a team of diverse image restoration models, not just single noise removers. This ensemble approach better tackles complex image artifacts. Crucially, ShaRP can sometimes learn to fix images even without needing perfect, clean examples. Our tests show ShaRP produces state-of-the-art results for tasks like MRI reconstruction and enhancing image resolution, outperforming current methods.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Optimization

Keywords: Inverse Problems, Image Reconstruction, Restoration Priors

Submission Number: 14452