A two-stream cnn based visual quality assessment method for light field images

Sana Alamgeer, Mylène C. Q. Farias

Abstract: Light Field (LF) cameras are able to capture both the intensity and the direction of light rays from the scene. This rich information demands a certain amount of memory and bandwidth for storage and transmission and, to alleviate this requirement, the LF content is processed and compressed. These operations often add degradations to the LF content that may affect their visual quality, requiring the use of methods to estimate the visual quality as perceived by the end consumer. In this paper, we propose a no-reference LF image quality assessment (LF-IQA) method that is based on a two-stream CNN architecture. The two-stream CNN extracts rich distortion-related spatial and angular binocular characteristics of LF contents to estimate their quality. More specifically, the first stream extracts angular information by processing Canny maps of Epipolar Plane Images (EPIs) generated from the corresponding LF contents, while the second stream extracts spatial information by processing mean canny maps generated from canny maps of sub-aperture images (SAIs). We also propose a novel approach to generate multiple epipolar-plane images - the MultiEPL. Results show that the proposed LF-IQA method outperforms state-of-the-art methods.