MS2DG-Net: Progressive Correspondence Learning via Multiple Sparse Semantics Dynamic Graph

Yizhang Liu, Luanyuan Dai

Abstract: Establishing superior-quality correspondences in an im- age pair is pivotal to many subsequent computer vision tasks. Using Euclidean distance between correspondences to find neighbors and extract local information is a com- mon strategy in previous works. However, most such works ignore similar sparse semantics information between two given images and cannot capture local topology among cor- respondences well. Therefore, to deal with the above prob- lems, Multiple Sparse Semantics Dynamic Graph Network (MS2DG-Net) is proposed, in this paper, to predict proba- bilities of correspondences as inliers and recover camera poses. MS2 DG-Net dynamically builds sparse semantics graphs based on sparse semantics similarity between two given images, to capture local topology among correspon- dences, while maintaining permutation-equivariant. Exten- sive experiments prove that MS2DG-Net outperforms state- of-the-art methods in outlier removal and camera pose es- timation tasks on the public datasets with heavy outliers. Source code:https://github.com/changcaiyang/MS2DG-Net