Real-Time Spoken Language Understanding for Orthopedic Clinical Training in Virtual Reality

Han Wei Ng, Aiden Koh, Anthea Foong, Jeremy Ong, Jun Hao Tan, Eng Tat Khoo, Gabriel Liu

Abstract: With the increasing limitation on healthcare resources, lack of real or standardized patients’ willingness to participate in medical education and a high medical litigation environment, the traditional learning of “see one” and “do one” is no longer acceptable. Virtual Reality training is becoming commonly used to address this issue. However, some main challenges such as the lack of a dynamic comprehensive conversation and poor language understanding prevents virtual reality from being adopted into mainstream education. In this study, a medical Natural Language Processing (NLP) pipeline is developed. The proposed pipeline displays a total accuracy of 95.6%, with significant improvements compared to the original baseline by an increase of 15% accuracy. The resultant NLP model is subsequently implemented into a medical clinical training virtual simulation.