Towards Multi-modal Self-supervised Video and Ultrasound Pose Estimation for Laparoscopic Liver Surgery

Nina Montaña Brown, João Ramalhinho, Bongjin Koo, Moustafa Allam, Brian R. Davidson, Kurinchi Gurusamy, Yipeng Hu, Matthew J. Clarkson

Abstract: Estimating a registration between intra-operative data and a pre-operative scan is a key step to enable image guidance, and is particularly challenging in the laparoscopic approach due to the limited field of views of the data sources in these interventions. In this paper, we propose the first multi-modal, self-supervised registration paradigm to perform simultaneous laparoscopic ultrasound and video alignment to CT of the liver. Preliminary experiments performed on a single, patient-specific anatomical CT model suggest that registration of multiple features can facilitate the alignment of both data sources, and we show an example registration on an instance of real clinical data.