Multiexposure Fused Light Field Image Quality Measurement in Dynamic Scenes Based on Joint Ghost Artifact and Spatial-Angular Representation Learning

Guanglong Liao, Gangyi Jiang, Yeyao Chen, Yueli Cui, Ting Luo, Haiyong Xu, Zhongjie Zhu, Mei Yu

Abstract: Multiexposure fused light field image (MEF-LFI) can record scene information with wide brightness range, thereby enhancing the performance and applications of light field imaging systems. However, quality degradation of MEF-LFI may be caused during the fusion process. Although numerous state-of-the-art image quality measurement methods have been proposed to predict image quality, only few of them consider the specific ghost artifacts and spatial-angular distortions associated with MEF-LFIs in dynamic scenes. To address this issue, this article proposes a novel MEF-LFI quality measurement method for dynamic scenes based on joint ghost artifact and spatial-angular representation learning. Specifically, assisted by the object segmentation-guided dynamic-static segmentation strategy, an isomeric transformer (ISOT) is designed to learn both dynamic and static region features in MEF-LFIs and establish semantic correlations between them. Furthermore, a spatial-angular feature learning (SAFL) module is constructed to learn multiscale color spatial information and angular information. Additionally, a novel salient attention mechanism is designed to enhance spatial texture detail extraction by mimicking the human visual perception. Finally, an adaptive pooling module along with three fully connected layers is constructed to integrate all features and measure the quality of MEF-LFIs. Experimental results indicate that the proposed method outperforms the state-of-the-art methods and is consistent with the human visual perception. Additionally, this work provides valuable insights for enhancing multiexposure fused light field imaging.