Modeling Scattering Effect for Under-Display Camera Image Restoration

Binbin Song, Jiantao Zhou, Xiangyu Chen, Shuning Xu

Abstract: The under-display camera (UDC) technology furnishes users with an uninterrupted full-screen viewing experience, eliminating the need for notches or punch holes. However, the translucent properties of the display lead to substantial degradation in UDC images. This work addresses the challenge of restoring UDC images by specifically targeting the scattering effect induced by the display. We explicitly model this scattering phenomenon by treating the display as a homogeneous scattering medium. Leveraging this physical model, the image formation pipeline is enhanced to synthesize more realistic UDC images alongside corresponding ground-truth images, thereby constructing a more accurate UDC dataset. To counteract the scattering effect in the restoration process, we propose a dual-branch network. The scattering branch employs channel-wise self-attention to estimate the scattering parameters, while the image branch capitalizes on the local feature representation capabilities of CNNs to restore the degraded UDC images. Additionally, we introduce a novel channel-wise cross-attention fusion block that integrates global scattering information into the image branch, facilitating improved restoration. To further refine the model, we design a dark channel regularization loss during training to reduce the gap between the dark channel distributions of the restored and ground-truth images. Comprehensive experiments conducted on both synthetic and real-world datasets demonstrate the superiority of our approach over current state-of-the-art UDC restoration methods. Our source code is publicly available at: https://github.com/NamecantbeNULL/SRUDC_pp.

External IDs:dblp:journals/ijcv/SongZCX25