Automatic 3D/2D Deformable Registration in Minimally Invasive Liver Resection using a Mesh Recovery Network

Mathieu Labrunie, Daniel Pizarro, Christophe Tilmant, Adrien Bartoli

Keywords: Mini-invasive Surgery, Liver, 3D/2D Registration, Augmented Reality

Abstract:

We propose the patient-specific Liver Mesh Recovery (LMR) framework, to automatically achieve Augmented Reality (AR) guidance by registering a preoperative 3D model in Minimally Invasive Liver Resection (MILR).
Existing methods solve registration in MILR by pose estimation followed with numerical optimisation and suffer from a prohibitive intraoperative runtime.
The proposed LMR is inspired by the recent Human Mesh Recovery (HMR) framework and forms the first learning-based method to solve registration in MILR.
In contrast to existing methods, the computation load in LMR occurs preoperatively, at training time.
We construct a patient-specific deformation model and generate patient-specific training data reproducing the typical defects of the automatically detected registration primitives.
Experimental results show that LMR's registration accuracy is on par with optimisation-based methods, whilst running in real-time intraoperatively.