Now You See Me: Recognizing the Player's Arousal Changes in the Game Through Game Footage Videos and Game Context Features

Yi Xia, Xiaoxu Li, Siyuan Chen, Ruck Thawonmas

Abstract: This paper proposes an approach that utilizes non-intrusive and non-restrictive multimodal data—game footage videos and game context features—to recognize the player’s arousal changes in the game. In recent years, affect modeling in games from a subject-agnostic perspective has emerged due to hardware limitations and ethical considerations. However, evaluation results of these approaches across various games indicate that their effectiveness is weaker in some games compared to other games. Design patterns in such games make it difficult for their methods to accurately capture the context of the games, thus making the task more challenging. Focusing on one of these games, we utilize a new preprocessing method to generate more samples from the game data in The Arousal video Game AnnotatIoN (AGAIN) dataset. In addition, we validate our hypothesis and evaluate the effectiveness of our proposed approach. We hypothesize that the information required to recognize the player’s arousal changes is embedded in the game footage videos and the game context features, and that this information can be more effectively learned by utilizing transfer learning with a model pre-trained on a large human action dataset, such as Kinetics-400. Experimental results demonstrate that our approach achieves an accuracy of 79.96% on the test set, which shows a significant improvement over existing methods on the AGAIN dataset. This suggests that our approach can be a valuable tool for affect modeling in games.