Multi-Motion Segmentation via Co-Attention-Induced Heterogeneous Model Fitting

Shuyuan Lin, Anjia Yang, Taotao Lai, Jian Weng, Hanzi Wang

Abstract: Motion segmentation is an essential task in artificial intelligence and computer vision. However, scene motion in real-world intelligent systems usually integrates multiple types of models, so specifying only one type of basic model may lead to the failure of scene-motion segmentation tasks. In this paper, we propose a novel and efficient heterogeneous model-fitting-based motion segmentation method (HMFMS) to accurately segment moving objects. HMFMS includes a new co-attention-induced heterogeneous model construction algorithm (HMC), an adaptive heterogeneous model refinement algorithm (HMR), and a heterogeneous model segmentation algorithm (HMS). First, we propose HMC to generate high-quality accumulated correlation matrices, by evaluating the quality of heterogeneous model hypotheses, based on the density estimation technique. Next, we propose HMR to construct sparse affinity matrices from the accumulated correlation matrices by applying information theory, effectively suppressing the values of correlations between different objects. Finally, we fuse the sparse affinity matrices and perform motion segmentation by using HMS, to obtain more accurate segmentation results. Experimental results show that HMFMS obtains superior performance on four challenging datasets (i.e., Hopkins155, Hopkins12, MTPV62 and KT3DMoSeg), compared with several subspace-based and model-fitting-based motion segmentation methods. More remarkably, HMFMS outperforms the state-of-the-art MCMS method by 57.1% and 1.8 times in terms of accuracy and computational efficiency on the representative KT3DMoSeg, respectively.