Rainmer: Learning Multi-view Representations for Comprehensive Image Deraining and Beyond

Wu Ran, Peirong Ma, Zhiquan He, Hong Lu

Abstract: We address image deraining under complex backgrounds, diverse rain scenarios, and varying illumination conditions, representing a highly practical and challenging problem. Our approach utilizes synthetic, real-world, and nighttime datasets, wherein rich backgrounds, multiple degradation types, and diverse illumination conditions coexist. The primary challenge in training models on these datasets arises from the discrepancies among them, potentially leading to conflicts or competition during the training period. To address this issue, we first align the distribution of synthetic, real-world and nighttime datasets. Then we propose a novel contrastive learning strategy to extract multi-view (multiple) representations that effectively capture image details, degradations, and illuminations, thereby facilitating training across all datasets. Regarding multiple representations as profitable prompts for deraining, we devise a prompting strategy to integrate them into the decoding process. This contributes to a potent deraining model, dubbed Rainmer. Additionally, a spatial-channel interaction module is introduced to fully exploit cues when extracting multi-view representations. Extensive experiments on synthetic, real-world, and nighttime datasets demonstrate that Rainmer outperforms current representative methods. Moreover, Rainmer achieves superior performance on the All-in-One image restoration dataset, underscoring its effectiveness. Furthermore, quantitative results reveal that Rainmer significantly improves object detection performance on both daytime and nighttime rainy datasets. These observations substantiate the potential of Rainmer for practical applications.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Content] Media Interpretation

Relevance To Conference: This work focuses on comprehensive image deraining using synthetic, real-world, and nighttime datasets, addressing a more practical and challenging problem. As a result, the technique proposed in this work enhances the visual quality of captured images under various rainy occasions, ambient lighting conditions, and multiple weather effects such as rain, raindrop, and snow. This improvement leads to enhanced performance in downstream applications, such as object detection.

Submission Number: 3488