Focal Modulation for Image Restoration

Yuning Cui, Wenqi Ren, Alois Knoll

Abstract: Image restoration aims to recover a sharp image from its degraded counterpart by removing degradations (e.g., noise, haze, and blur) and restoring missing details. It plays an important role in many fields, such as remote sensing and medical imaging. How to effectively capture critical information parsimoniously for high-quality reconstruction has long been a pivotal problem in this domain. This study aims to develop an efficient and effective focal modulation scheme for image restoration. Inspired by the fact that different regions in a corrupted image always undergo degradations in various degrees, we introduce a dual-domain selection mechanism to emphasize crucial information for restoration, such as edge signals and hard regions. Moreover, a channel modulation module is developed to facilitate channel interactions by exploring the utility of the Fourier transform in channel dimensions. In addition, we split high-resolution features to insert multi-scale receptive fields into the network, improving efficiency and performance. Incorporating these designs into a U-shaped convolutional backbone, the network achieves state-of-the-art performance on 13 different datasets for five general image restoration tasks, including dehazing, desnowing, deraining, motion/defocus deblurring, and low-light enhancement. To further demonstrate the effectiveness of our focal modulation strategy, we apply it to the all-in-one image restoration setting, and the obtained model performs favorably against state-of-the-art all-in-one algorithms. Moreover, our module extends effectively to tasks such as composite degradation, medical imaging, and ultra-high-definition image restoration.

External IDs:dblp:journals/ijcv/CuiRK26