IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models

Hanting Wang, Tao Jin, Wang Lin, Shulei Wang, Hai Huang, Shengpeng Ji, Zhou Zhao

TL;DR: We propose a noval framework that enables leveraging existing powerful generative priors in image restoration bridges.

Abstract: Bridge models in image restoration construct a diffusion process from degraded to clear images. However, existing methods typically require training a bridge model from scratch for each specific type of degradation, resulting in high computational costs and limited performance. This work aims to efficiently leverage pretrained generative priors within existing image restoration bridges to eliminate this requirement. The main challenge is that standard generative models are typically designed for a diffusion process that starts from pure noise, while restoration tasks begin with a low-quality image, resulting in a mismatch in the state distributions between the two processes. To address this challenge, we propose a transition equation that bridges two diffusion processes with the same endpoint distribution. Based on this, we introduce the **IRBridge** framework, which enables the direct utilization of generative models within image restoration bridges, offering a more flexible and adaptable approach to image restoration. Extensive experiments on six image restoration tasks demonstrate that IRBridge efficiently integrates generative priors, resulting in improved robustness and generalization performance. Code will be available at GitHub.

Lay Summary:

Existing image restoration bridge models often require training a dedicated model from scratch for each specific restoration task, which is typically costly and time-consuming. In this work, we explore whether pretrained generative models can be leveraged to address this limitation, as they also aim to produce high-quality images as outputs.

The main challenge lies in the fact that bridge models define a diffusion process that differs from that of standard generative models, resulting in mismatched state distributions along the process. To overcome this, we propose a transition equation that enables state transformation between two diffusion processes sharing the same endpoint distribution. Building on this, we introduce a new framework, IRBridge, which effectively incorporates generative priors by reusing powerful pretrained image generative models, reducing training costs while enhancing restoration performance.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Applications->Computer Vision

Keywords: Image Restoration, Diffusion Models, Bridge Models

Submission Number: 2770