IMP: Iterative Matching and Pose Estimation with Adaptive Pooling

Fei Xue, Ignas Budvytis, Roberto Cipolla

Abstract:

Previous methods solve feature matching and pose estimation using a two-stage process by first finding matches
and then estimating the pose. As they ignore the geometric relationships between the two tasks, they focus on either improving the quality of matches or filtering potential outliers, leading to limited efficiency or accuracy. In
contrast, we propose an iterative matching and pose estimation framework (IMP) leveraging the geometric connections between the two tasks: a few good matches are
enough for a roughly accurate pose estimation; a roughly
accurate pose can be used to guide the matching by providing geometric constraints. To this end, we implement
a geometry-aware recurrent attention-based module which
jointly outputs sparse matches and camera poses. Specifically, for each iteration, we first implicitly embed geometric information into the module via a pose-consistency
loss, allowing it to predict geometry-aware matches progressively. Second, we introduce an efficient IMP, called
EIMP, to dynamically discard keypoints without potential
matches, avoiding redundant updating and significantly reducing the quadratic time complexity of attention computation in transformers. Experiments on YFCC100m, Scannet,
and Aachen Day-Night datasets demonstrate that the proposed method outperforms previous approaches in terms
of accuracy and efficiency. Code is available at https:
//github.com/feixue94/imp-release