Unified Removal of Raindrops and Reflections: A New Benchmark and A Novel Pipeline

Xingyu Liu, Zewei He, Yu Chen, Chunyu Zhu, Zixuan Chen, Xing Luo, Zhe-Ming Lu

Keywords: New benchmark, Image restoration, Raindrops and Reflections

Abstract: When capturing images through glass surfaces or windshields on rainy days, raindrops and reflections frequently co-occur to significantly reduce the visibility of captured images. Prior de-raindrop and de-reflection studies have failed to simultaneously remove both types of degradations from a single captured image, thereby limiting their application and robustness in real-world scenarios. In this work, we make the first attempt to explore this new task, \ie, unified removal of raindrops and reflections (UR$^3$). First of all, we set up an image acquisition platform to collect our own dataset, namely RainDrop and ReFlection (RDRF) dataset, which provides a new benchmark with substantial, high-quality, diverse image pairs. Within each pair, one has a clean foreground and the rest is corrupted by raindrops and reflections. Second, we propose a diffusion-based framework (\ie, DiffUR$^3$) to decouple the UR$^3$ task into a restoration stage and a conditional generation stage (with multiple conditions). By leveraging the powerful generative prior, DiffUR$^3$ successfully removes both degradations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on our benchmark and on challenging in-the-wild images. The RDRF dataset and the codes will be made public upon acceptance.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 6323