Surgical Gesture Recognition in Robot-Assisted Surgery using Machine Learning Methods on Kinematic Data

Alexandros Dimitriadis, George Moustris, Costas S. Tzafestas

Keywords: Surgical Gesture Recognition, Robotic Surgery, JIGSAWS, LSTM, Kinematic Data, Real-time, Machine Learning, Hybrid Model, Attention Mechanism, Self Attention, CRF

TL;DR: This study proposes hybrid neural network models, especially those incorporating attention layers, for real-time surgical gesture recognition from kinematic data, achieving an 81.56% accuracy on the JIGSAWS dataset and outperforming existing methods.

Abstract: This work focuses on training machine learning models to recognize gestures during robot-assisted surgical procedures in real-time, using exclusively kinematic data from the patient-side manipulators. The JIGSAWS dataset, specifically the suturing tasks, serves as the evaluation benchmark. We experimented with various neural network architectures, using an LSTM architecture as the baseline approach. To further enhance performance, two hybrid approaches are proposed in this work: the first one combining an LSTM with a Conditional Random Field (CRF) and the second one integrating an attention layer. An extensive experimental study was conducted to evaluate and optimize the performance of the different approaches, and identify areas for improvement. A thorough comparative analysis of the results shows that the proposed hybrid approaches, in particular the one combining an attention layer, can improve recognition rate, as compared to relevant state-of-the-art, with accuracy of 81.56\%. This study lays the foundation for further research in the field, focusing on advancing real-time surgical gesture recognition as a means to develop tools that can provide intraoperative monitoring and assistance to the surgeon.

Submission Number: 131