Gesture Evaluation in Virtual Reality

Anna Deichler, Jonas Beskow, Axel Wiebe Werner

Keywords: virtual reality, gesture generation, embodied conversational agents, evaluation paradigms, dyadic interaction

Abstract: Gestures play a crucial role in human communication, enhancing interpersonal interactions through non-verbal expression. Burgeon- ing technology allows virtual avatars to leverage communicative gestures to enhance their life-likeness and communication quality with AI-generated gestures. Traditionally, evaluations of AI-generated gestures have been confined to 2D settings. However, Virtual Reality (VR) offers an immersive alternative with the potential to affect the perception of virtual gestures. This paper introduces a novel evaluation approach of computer-generated gestures, investigating the impact of a fully immersive environment in comparison to a traditional 2D setting. The goal is to find the differences, benefits, and drawbacks of the two alternatives. Furthermore, the study also aims to investigate three gesture generation algorithms submitted to the 2023 Genea Challenge and evaluate their performance in the two virtual settings. Experiments showed that the VR setting has an impact on the rating of generated gestures. Participants tended to rate gestures observed in VR slightly higher on average than in 2D. Furthermore, the results of the study showed that the generation models used for the study had a consistent ranking. However, the setting had a limited impact on the models’ performance, having a bigger impact on the perception of ’true movement’ which had higher ratings in VR than in 2D.

Submission Number: 5