RoSe: Rotation-Invariant Sequence-Aware Consensus for Robust Correspondence Pruning

Yizhang Liu, Weiwei Zhou, Yanping Li, Shengjie Zhao

Abstract: Correspondence pruning has recently drawn considerable attention as a crucial step in image matching. Existing methods typically achieve this by constructing neighborhoods for each feature point and imposing neighborhood consistency. However, the nearest-neighbor matching strategy often results in numerous many-to-one correspondences, thereby reducing the reliability of neighborhood information. Furthermore, the smoothness constraint fails in cases of large-scale rotations, leading to misjudgments. To address the above issues, this paper proposes a novel robust correspondence pruning method termed RoSe, which is based on rotation-invariant sequence-aware consensus. We formulate the correspondence pruning problem as a mathematical optimization problem and derive a closed-form solution. Specifically, we devise a rectified local neighborhood construction strategy that effectively enlarges the distribution between inliers and outliers. Meanwhile, to accommodate large-scale rotation, we propose a relative sequence-aware consistency as an alternative to existing smoothness constraints, which can better characterize the topological structure of inliers. Experimental results on image matching and registration tasks demonstrate the effectiveness of our method. Robustness analysis involving diverse feature descriptors and varying rotation degrees further showcases the efficacy of our method.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: This work significantly contributes to multimedia and multimodal processing by addressing key challenges in image matching, specifically in correspondence pruning. By introducing the RoSe method, which leverages rotation-invariant sequence-aware consensus, the paper offers a robust solution to enhance the reliability of neighborhood information. The proposed method effectively deals with the limitations of existing approaches, such as the many-to-one correspondences caused by nearest-neighbor matching and the failure of smoothness constraints under large-scale rotations. Experimental results on image matching and registration tasks demonstrate the effectiveness of our method, indicating its applicability in multimedia processing. Alation studies involving diverse feature descriptors and varying rotation degrees further showcase the robustness of our method and the potential for handling various multimedia datasets and scenarios. Overall, this work provides a valuable contribution to advancing multimedia and multimodal processing techniques, particularly in the realm of image analysis and matching.

Submission Number: 2672