Investigating keypoint descriptors for camera relocalization in endoscopy surgery

Isabela Hernández, Roger Soberanis-Mukul, Jan Mangulabnan, Manish Sahu, Jonas Winter, S. Swaroop Vedula, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath

Abstract: Recent advances in computer vision and machine learning have resulted in endoscopic video-based solutions for dense reconstruction of the anatomy. To effectively use these systems in surgical navigation, a reliable image-based technique is required to constantly track the endoscopic camera’s position within the anatomy, despite frequent removal and re-insertion. In this work, we investigate the use of recent learning-based keypoint descriptors for six degree-of-freedom camera pose estimation in intraoperative endoscopic sequences and under changes in anatomy due to surgical resection.