FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining
Abstract: Images corrupted by rain streaks often lose vital frequency information for perception, and image deraining aims to solve this issue which relies on global and local degradation modeling.
Recent studies have witnessed the effectiveness and efficiency of Mamba for perceiving global and local information based on its exploiting local correlation among patches, however, rarely attempts have been explored to extend it with frequency analysis for image deraining, limiting its ability to perceive global degradation that is relevant to frequency modeling (e.g. Fourier transform).
In this paper, we propose FreqMamba, an effective and efficient paradigm that leverages the complementary between Mamba and frequency analysis for image deraining. The core of our method lies in extending Mamba with frequency analysis from two perspectives: extending it with frequency-band for exploiting frequency correlation, and connecting it with Fourier transform for global degradation modeling.
Specifically, FreqMamba introduces complementary triple interaction structures including spatial Mamba, frequency band Mamba, and Fourier global modeling. Frequency band Mamba decomposes the image into sub-bands of different frequencies to allow 2D scanning from the frequency dimension. Furthermore, leveraging Mamba's unique data-dependent properties, we use rainy images at different scales to provide degradation priors to the network, thereby facilitating efficient training. Extensive experiments show that our method outperforms state-of-the-art methods both visually and quantitatively.
Primary Subject Area: [Content] Vision and Language
Relevance To Conference: From the perspective of global modeling, this paper examines the limitations of Fourier transform and Mamba in image deraining tasks, and establishes a seamless relationship between them by introducing the frequency dimension Mamba . We propose FreqMamba, a general and efficient paradigm that leverages the unique and complementary capabilities of frequency analysis and Mamba for image deraining.
Supplementary Material: zip
Submission Number: 657
Loading