FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining

Dong Li, Yidi Liu, Xueyang Fu, Jie Huang, Senyan Xu, Qi Zhu, Zheng-Jun Zha

Abstract: Image deraining aims to remove rain streaks from rainy images and restore clear backgrounds. Currently, some research that employs the Fourier transform has proved to be effective for image deraining, due to it acting as an effective frequency prior for capturing rain streaks. However, despite there exists dependency of low frequency and high frequency in images, these Fourier-based methods rarely exploit the correlation of different frequencies for conjuncting their learning procedures, limiting the full utilization of frequency information for image deraining. Alternatively, the recently emerged Mamba technique depicts its effectiveness and efficiency for modeling correlation in various domains (e.g., spatial, temporal), and we argue that introducing Mamba into its unexplored Fourier spaces to correlate different frequencies would help improve image deraining. This motivates us to propose a new framework termed FourierMamba, which performs image deraining with Mamba in the Fourier space. Owing to the unique arrangement of frequency orders in Fourier space, the core of FourierMamba lies in the scanning encoding of different frequencies, where the low-high frequency order formats exhibit differently in the spatial dimension (unarranged in axis) and channel dimension (arranged in axis). Therefore, we design FourierMamba that correlates Fourier space information in the spatial and channel dimensions with distinct designs. Specifically, in the spatial dimension Fourier space, we introduce the zigzag coding to scan the frequencies to rearrange the orders from low to high frequencies, thereby orderly correlating the connections between frequencies; in the channel dimension Fourier space with arranged orders of frequencies in axis, we can directly use Mamba to perform frequency correlation and improve the channel information representation. Extensive experiments reveal that our method outperforms state-of-the-art methods both qualitatively and quantitatively.

Lay Summary:

Rain in photos can seriously degrade image quality, which creates challenges for applications like autonomous driving or outdoor surveillance. Traditional computer vision methods try to clean these images by analyzing them in either the pixel space (what we directly see) or the frequency space (a mathematical way of looking at how fast things change in an image). Recent research shows that using the frequency space helps remove rain better, since rain streaks have special frequency patterns. But current methods often ignore the relationships between different types of frequencies — like low (broad shapes) and high (fine details).
In our research, we apply a powerful new machine learning tool called Mamba to connect these frequencies more effectively. Our method, called FourierMamba, explores two different ways of organizing and scanning the frequencies, depending on how they appear in the image. This allows the model to better learn how different frequency patterns relate to rain.
The result? Our approach produces cleaner images than existing methods — potentially making rainy-day vision systems smarter and safer.

Primary Area: Applications->Computer Vision

Keywords: Fourier, deraining, Mamba

Submission Number: 9589