Single Image Reflection Removal From Glass Surfaces via Multi-Scale Reflection Detection

Tao Yan, Helong Li, Jiahui Gao, Zhengtian Wu, Rynson W. H. Lau

Abstract: While people take photos through glass surfaces with consumer imaging equipments, such as smartphones or digital cameras, reflections on the glass surfaces of captured images cannot be avoided. Reflections not only affect the aesthetic perception of captured images but also degrade the performance of subsequent tasks. Since the prominence of reflections depends on the contrast between the foreground reflection layer and the background transmission layer, reflections on different regions of the same glass surface may have extremely different sizes and brightness contrasts. In this paper, we propose a reflection removal network, named MRDNet, with a novel multi-scale reflection detector for foreground reflection detection and a novel Multi-scale Reflection-aware Transmission Recovery (MRTR) module for background transmission layer restoration. Our MRDNet has three main modules. First, two parallel feature extraction modules, called Reflection Extractor (R-Extractor) and Transmission Extractor (T-Extractor), are introduced to extract features of the foreground reflection layer and features of the background transmission layer contaminated by reflections, from the input image, respectively. Then, a Multi-scale Reflection-aware Transmission Recovery (MRTR) module is proposed to recover textural details of the background transmission layer. For learning our network, we also propose a large scale 3D virtual-world Physical Rendering-based Reflection Removal (PRGR) dataset.