SFIR: Optimizing spatial and frequency domains for image restoration

Open Webpage

Yubin Gu, Yuan Meng, Siting Chen, Jiayi Ji, Xiaoshuai Sun, Weijian Ruan, Rongrong Ji

Published: 01 Jan 2026, Last Modified: 12 Nov 2025Pattern Recognit. 2026EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Image restoration (IR) plays a pivotal role in image processing, significantly influencing subsequent computer vision tasks. Recent advancements have highlighted the efficacy of deep neural networks in enhancing image restoration. Yet, these methods often underutilize critical spatial and frequency domain information, thereby constraining their performance. Specifically, Transformer-based techniques, despite bolstering global feature associations in the spatial domain, incur high computational costs and yield marginal performance gains. Addressing these limitations, we introduce SFIR, a novel convolutional network model designed for comprehensive image degradation correction. SFIR incorporates an encoder-decoder architecture, with its core innovation rooted in two modules: the Multi-scale Spatial Enhancement (MSE) module and the Frequency Amplitude Modulation (FAM) module. The MSE module enhances the model’s ability to harness spatial domain information. It achieves this by combining multi-scale feature fusion with sophisticated local window-based non-local attention mechanisms. In addition, MSE also includes an adaptive detail enhancement block, which further optimizes the model’s ability to handle complex degradations by refining detail features and smoothing features. On the other hand, the FAM module effectively addresses the challenges of substantial amplitude map variances across various channels. It uses dynamic weighting modulation to improve image clarity and restore texture details. Collectively, these modules enhance SFIR’s efficiency in leveraging both spatial and frequency domain insights, leading to notable restoration quality improvements. Experimental validation underscores SFIR’s superior performance in deraining, dehazing, and particularly deblurring, while also showcasing its advantages in parameter efficiency and computational cost. Besides, we also released the source code at https://github.com/ClimBin/SFIR.