Emotion Recognition from Eye Movements Using Multi-way Autoregressive Model

Tian-Fang Ma, Xuan-Hao Liu, Wei-Long Zheng, Bao-Liang Lu

Abstract: The application of physiological signals in emotion recognition is a popular research topic in human-computer interactions. Eye movement, as an important physiological signal, plays an essential role in medicine, psychology, cognitive science, and other scientific research fields. Previous studies have successfully identified human emotions by combining various eye-related measurements, selecting features, and utilizing machine learning techniques. However, the exploration of eye movement signals in emotion recognition remains insufficient. In this study, we utilize eye tracking heatmap and eye movement trajectory data for emotion recognition for the first time. Based on the two types of eye movement data, we develop a multi-way autoregressive model capable of processing multi-view eye movement data. Compared to traditional deep learning baseline models, our model better adapts to the structure of eye movement data and significantly improves the classification performance. Furthermore, we integrate heatmap and trajectory data with commonly used eye-related measurement features, which further enhance the performance of emotion recognition beyond previous methods.