Multi-modal classification of cognitive load in a VR-based training system

Srikrishna S. Bhat, Chelsea Dobbins, Arindam Dey, Ojaswa Sharma

Abstract: Training systems are used in many industries, ranging from surgery to space missions to rehabilitation. Virtual Reality (VR) is a technology that has been incorporated as an effective tool in such training systems to simulate the environment, especially in situations where the training can’t take place in the actual environment. For a training environment and task to be effective, it must sufficiently challenge the trainee. One parameter that can be used to measure this is cognitive load (CL), which is defined as the amount of working memory used while performing a learning task. This parameter needs to be sufficiently high to maximize learning but not too high as to overload the trainee. However, the challenge is to detect this state using objective physiological measures, which can be collected during the entire task. This paper presents a study to classify CL using a combination of Electroencephalogram (EEG) and Electrodermal Activity (EDA) signals during a procedural VR training task. Thirty participants undertook a study where they built a designated model within a given time over multiple levels that were constructed to induce low to high CL. Features generated from the data were subject to feature selection (FS), which was undertaken using the Mutual Information (MI) technique. Binary classification models were developed using Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (kNN), Extreme Gradient Boosting (Xgboost) and Multi-Layer Perceptrons (MLP). Results illustrated that the Xgboost classifier performed the best with an F1-score of $0.831 \pm 0.030$ and accuracy of $0.805 \pm 0.033.$ SHAP analysis of the features illustrated greater contributions from the frontal and occipital regions of the brain and frequency domain features from tonic skin conductance.