Predicting Empathy and Other Mental States During VR Sessions Using Sensor Data and Machine Learning

Emilija Kizhevska, Hristijan Gjoreski, Mitja Luštrek

Abstract: Virtual reality (VR) is often regarded as the “ultimate empathy machine” because of its ability to immerse users in alternative perspectives and environments beyond physical reality. In this study, 105 participants (average age 22.43 ± 5.31 years, range 19–45, 75% female) with diverse educational and professional backgrounds experienced three-dimensional 360° VR videos featuring actors expressing different emotions. Despite the availability of established methodologies in both research and clinical domains, there remains a lack of a universally accepted “gold standard” for empathy assessment. The primary objective was to explore the relationship between the empathy levels of the participants and the changes in their physiological responses. Empathy levels were self-reported using questionnaires, while physiological attributes were recorded through various sensors. The main outcomes of the study are machine learning (ML) models capable of predicting state empathy levels and trait empathy scores during VR video exposure. The Random Forest (RF) regressor achieved the best performance for trait empathy prediction, with a mean absolute percentage error (MAPE) of 9.1%, and a standard error of the mean (SEM) of 0.32% across folds. For classifying state empathy, the RF classifier achieved the highest balanced accuracy of 67%, and a standard error of the proportion (SE) of 1.90% across folds. This study contributes to empathy research by introducing an objective and efficient method for predicting empathy levels using physiological signals, demonstrating the potential of ML models to complement self-reports. Moreover, by providing a novel dataset of VR empathy-eliciting videos, the work offers valuable resources for future research and clinical applications. Additionally, predictive models were developed to detect non-empathic arousal (78% balanced accuracy ± 0.63% SE) and to distinguish empathic vs. non-empathic arousal (79% balanced accuracy ± 0.41% SE). Furthermore, statistical tests explored the influence of narrative context, as well as empathy differences toward different genders and emotions. We also make available a set of carefully designed and recorded VR videos specifically created to evoke empathy while minimizing biases and subjective perspectives.

External IDs:doi:10.3390/s25185766