Editorial: Virtual agents in virtual reality: design and implications for VR users

Pierre Raimbaud, Béatrice Biancardi, Iana Podkosova, Catherine Pelachaud

Abstract: The following research topic has emerged in a context of expansion of virtual communication and virtual social links in our society. At the same time, the enhancement of artificial intelligence technologies and autonomous agents designs has facilitating for populating Virtual Environments (VE) with not only real users, but also with more and more believable Virtual Human Agents (VHA) Demeure et al. (2011).This research topic focuses on Virtual Reality (VR), virtual agents design and its effect on VR users. Previous research has shown that the content provided to users in VR do have an effect on them, mainly due to the immersion and sense of presence that VEs can induce Slater et al. (1996). Realism and believability of the environments designed for being used in VR technologies is felt by the users on usually at least their visual perception and interaction capabilities, sometimes more. Indeed, other multisensory perceptions have been explored in the literature, such as touch Hoppe et al. (2020), sound Griol et al. (2019) or even smell Javerliat et al. (2022), which are constantly improving. This empowers VR with the ability to enable the replication of real-world human behaviours, as an ecological environment where real situations can be mimicked and elicit realistic reactions on users. These reactions can be evaluated either through self-reported assessments or physiological measures during the experiment. Relying on the social aspect of human life, our topic aimed to gather research studies that proposed VR experiments with social situations between users and virtual agents Von der Pütten et al. (2009). We wondered here which aspects of virtual agents design could affect VR users behaviours, in which manner, and how VHAs are perceived. Four articles have been published, all relying on user experiments: two propose an evaluation of the effect of specific agents behaviours on VR users actions or reactions, and two design a new dataset of virtual agents and rate their perception by users.Bönsch et al, 2024 propose an innovative approach of guidance towards points of interests in a virtual environment, through social behaviours of a virtual crowd of humans. In this paper, two modalities are explored, compared to a control condition with a virtual crowd with no guidance actions: i) the crowd of agents actively supports the user towards the points of interests, and ii) the crowd implicitly induces users towards the goals, as a more passive flow such as walking in a specific direction. Results show that the active support modality is found as more effective, nonetheless the authors note that the implicit flow of agents could be promising for specific conditions (large streets, "leader" user personality etc.). Authors' results support for the use of agents as guides in VR, through visual but also audio stimuli, reinforcing the interest of multisensory for the design of VR experiments with virtual agents.Ban et al, 2024's article originality relies on the kind of measurement they used: a physiological measure on the saliva, to qualify the autonomic nervous system reaction of VR users facing stressful situations. In their experiment, in a customer service training, VR users are sequentially exposed to virtual agents with high-intensity stressor and high-intensity one. The authors succeed to show similar results compared to previous studies in both real-life and VR, conducted before with a different user task than here -the trier social stress test. This shows the ecological validity of their VR environment to elicit physiological reactions due to a quantity of induced stress, and the correctness of their evaluation through the objective measurements they use. Future work would be to evaluate which factors of VHAs' behaviours (gestures, voice tone) have a greater influence on users' stress responses.Siehl et al, 2024 create a new publicly available dataset of animated virtual humanoid male characters, reusing and adapting a generator of 2D faces ; they evaluate their emotional effect on users in terms of trustworthiness, valence and arousal. Their results show that manipulations on the animations that were expected to accordingly influence the trustworthiness have indeed modified this emotional perception, verified here through a first user study relying on video stimuli. Moreover, the authors have conducted another study with additional audio stimuli to accentuate the increase or decrease of trustworthiness towards users, which have produced the same results, even with a stronger effect. Future improvements mainly remain on adapting the social context of such virtual characters animations when displayed to users, for a more in-context evaluation of this VHA dataset. Do et al, 2023's article offers multiple contributions: i) the creation of a new rich and diverse dataset of 210 fully rigged avatars, with high racial diversity and inclusion; ii) a precise and complete methodology for the creation of such kind of dataset (modelling process, recruitment strategy of VHA designers and community/race representatives, iterations, user evaluations); iii) knowledge about the perception of the dataset itself and more generally of virtual agents of different races by a very diverse panel of users. In this study, the identification of VHAs' race by the users that have declared themselves as pertaining to the same race as the one of the VHA displayed is very accurate. Moreover, Asian, Black, and White VHA's are recognized by all participants, whereas Hispanic and MENA VHAs are only validated as such by users who have identified themselves as belonging to these groups. Future work could also be done on variety inside groups, e.g. to represent the diversity of Hispanic profiles across Latin America, or MENA or NHPI cultural diversity, which can be directly reflected on their style, e.g., hair cuts.Through these articles, this research topic contributes to the diffusion of public datasets of virtual human agents, to the explanation of design methodology and evaluation approach on virtual agents, and to the understanding of human-agent perception and interactions in VR. Future studies could contribute to extend the development of multisensory aspects in the design of virtual agents behaviours, in line with the audio features explored here, and on the in-context user evaluations of social VHAs, through subjective and objective measurements such as physiological data.