Efficient Segmentation-Based PatchMatch for Large Displacement Optical Flow Estimation

Jun Chen, Zemin Cai, Jianhuang Lai, Xiaohua Xie

Abstract: Efficient optical flow estimation with high accuracy is a challenging problem in computer vision. In this paper, we present a simple but efficient segmentation-based PatchMatch framework to address this issue. Specifically, it firstly generates sparse seeds without losing important motion information by over-segmentation, and then yields sparse matches by adopting a coarse-to-fine PatchMatch with sparse seeds. Such a scheme enhances the robustness of global regularization and yields better matching results compared with the existing NNF techniques while leading to a significant speed-up due to the sparsity of these seeds. Simultaneously, we introduce an extended nonlocal propagation and adaptive random search to address the basic limitation of the traditional coarse-to-fine framework in handing motion details that often vanish at coarser levels. Finally, we obtain dense matches at the finest level through an efficient sparse-to-dense matching according to the cues of over-segmentation. While performing an efficient approximation for over-segmentation, the proposed algorithm runs significantly fast and is robust to large displacements while preserving important motion details. It also achieves good performance on the challenging MPI-Sintel and Kitti flow 2015 datasets.