Automatic Facial Landmark Detection for Neurosurgical Mixed Reality Applications in MRI and CT scans using Deep Learning

Mathijs de Boer, Lambertus W. Bartels, Pierre A.J.T. Robe, Tristan P.C. van Doormaal

Keywords: Deep Learning, Facial Landmark Detection, Geometry, Mixed Reality, Neurosurgery

Abstract: Mixed reality (MxR) has the potential to revolutionize the way neurosurgical interventions are performed. However, the use of MxR in the operating room (OR) introduces new challenges, such as the registration of preoperative images to the patient. This paper presents a deep learning method for automatic detection of facial landmarks in Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) scans, which can be used for image-to-patient registration in MxR applications. The method achieves a mean error of 4.02(±2.65) mm in 3.55(±1.53) seconds on a CPU. Apart from the nasion, no statistically significant differences were found between the performance of the method between the CT and MRI scans.

Submission Number: 22