XRefine - Attention-Guided Keypoint Match Refinement

Jan Fabian Schmid, Annika Hagemann

Keywords: Feature Detection, Match Refinement, Deep Learning, Camera Pose Estimation, Structure From Motion

TL;DR: XRefine is a detector-agnostic method that uses cross-attention on image patches to refine keypoint matches and enhance 3D vision accuracy across datasets.

Abstract: Sparse keypoint matching is crucial for 3D vision tasks, yet current keypoint detectors often produce spatially inaccurate matches. Existing refinement methods mitigate this issue through alignment of matched keypoint locations, but they are typically detector-specific, requiring retraining for each keypoint detector. We introduce XRefine, a novel, detector-agnostic approach for sub-pixel keypoint refinement that operates solely on image patches centered at matched keypoints. Our cross-attention-based architecture learns to predict refined keypoint coordinates without relying on internal detector representations, enabling generalization across detectors. Furthermore, XRefine can be extended to handle multi-view feature tracks. Experiments on MegaDepth, KITTI, and ScanNet demonstrate that the approach consistently improves geometric estimation accuracy, achieving superior performance compared to existing refinement methods while maintaining runtime efficiency. Our code and trained models can be found at https://github.com/boschresearch/xrefine.

Submission Number: 187