Frequency-Guided Posterior Sampling for Diffusion-Based Image Restoration

Darshan Thaker, Abhishek Goyal, Rene Vidal

Abstract:

Image restoration aims to recover high-quality images from
degraded observations. When the degradation process is
known, the recovery problem can be formulated as an inverse problem, and in a Bayesian context, the goal is to sample a clean reconstruction given the degraded observation.
Recently, modern pretrained diffusion models have been
used for image restoration by modifying their sampling procedure to account for the degradation process. However,
these methods often rely on certain approximations that can
lead to significant errors and compromised sample quality.
In this paper, we propose a simple modification to existing
diffusion-based restoration methods that exploits the frequency structure of the reverse diffusion process. Specifically, our approach, denoted as Frequency Guided Posterior Sampling (FGPS), introduces a time-varying lowpass filter in the frequency domain of the measurements,
progressively incorporating higher frequencies during the
restoration process. We provide the first rigorous analysis of the approximation error of FGPS for linear inverse
problems under distributional assumptions on the space of
natural images, demonstrating cases where previous works
can fail dramatically. On real-world data, we develop an
adaptive curriculum for our method’s frequency schedule
based on the underlying data distribution. FGPS significantly improves performance on challenging image restoration tasks including motion deblurring and image dehazing.