Exploring Matching Rates: From Key Point Selection to Camera Relocalization

Hu Lin, Chengjiang Long, Yifeng Fei, qianchen xia, Erwei Yin, Baocai Yin, Xin Yang

Abstract: Camera relocalization is the task of estimating camera pose within a known scene. It has important applications in the fields of Virtual Reality (VR), Augmented Reality (AR), robotics, and more within the domain of computer vision. Learning-based camera relocalizers have demonstrated leading pose accuracy, yet all current methods invariably utilize all the information within an image for pose estimation. This may offer robustness under challenging viewpoints but impacts the localization accuracy for viewpoints that are easier to localize. In this paper, we propose a method to gauge the credibility of image pose, enabling our approach to achieve more accurate localization on keyframes. Additionally, we have devised a keypoint selection method predicated on matching rate. Furthermore, we have developed a keypoint evaluation technique based on reprojection error, which estimates the scene coordinates for points within the scene that truly warrant attention, thereby enhancing the localization performance for keyframes. We also introduce a gated camera pose estimation strategy, employing an updated keypoint-based network for keyframes with higher credibility and a more robust network for difficult viewpoints. By adopting an effective curriculum learning scheme, we have achieved higher accuracy within a training span of just 20 minutes. Our method's superior performance is validated through rigorous experimentation. The code will be released.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Experience] Interactions and Quality of Experience

Relevance To Conference: In this work, we focus on investigating the reliability and accuracy of visual relocalization within the multimedia domains of Augmented Reality (AR), Virtual Reality (VR), and robotics. By analyzing and processing image data, we are able to select images with a credible pose estimation, which is crucial for enhancing the user experience in practical applications.

Submission Number: 5138