Light field scene flow with occlusion regularization

Jie Chen, Yun Ni, Junhui Hou, Lap-Pui Chau

Abstract: The scene flow provides a comprehensive understanding of the vision field's 3D dynamics, which is extremely useful for most computer vision applications. Stereo vision has been the conventional means for scene flow estimation. However, light field camera provides a more convenient and reliable solution for the same task with its unique advantage in scene depth estimation. In this work, we propose a joint estimation framework for the optical flow and depth flow given two light field images as input. The scene depth estimated from the light field will be used for occlusion detection and sparse correspondence regularization which results in a more robust estimation of optical flow. Depth flow will be calculated based on the interpolated correspondence matching. Experiments show that the proposed framework can produce competitive scene flow estimation at a much lower computational cost, compared to state-of-the-art methods.