Adaptive Threshold Mask Prediction and Occlusion-aware Convolution for Foreground Occlusions in Light Fields

Jieyu Chen, Ping An, Xinpeng Huang, Chao Yang

Abstract: The performance of existing de-occlusion methods is limited mainly due to inaccurate foreground mask prediction, interference from occlusion information in feature extractors, and insufficient utilization of sub-pixel information between views. Therefore, in this paper, we propose an efficient light field image de-occlusion method to improve the performance of occlusion localization and occlusion removal. First, we design an adaptive threshold mask prediction branch, which fully utilizes the spatial and angular information of light field images and incorporates mask binarization into the network for joint optimization. Then, we propose Occlusion-aware Convolution, which can more efficiently extract the joint spatial-angular features of the occluded light field image. Additionally, we design a sub-pixel complement strategy. This strategy fully utilizes the sub-pixel information between the views and supplements it into the view to be deoccluded. Experimental results demonstrate that our method achieves superior performance on both real-world and synthetic light field datasets.