Who and Where Am I? Embodied Cognition-Aware Virtual Humans

Fan Lu, Wentao Zhu, Kwan-Yee Lin, Wayne Wu, Yizhou Wang, Guang Chen

Keywords: virtual human, embodied cognition

Abstract: Building virtual humans requires more than just realistic appearances and diverse motions; it necessitates simulating the intricate interplay between internal cognitive states, external environments, and executed motion behavior, as framed by the concept of embodied cognition. In this paper, we propose an embodied cognitive architecture, EmbodiedHuman, that captures this interaction by integrating "Mind" - a structured cognitive module, with motor execution to drive the virtual human’s behavior within an interactive 3D environment. To enable integrated embodiment over both cognitive states and physical execution, we introduce three novel modules in a unified framework: $1)$ a cognition-inspired Mind structure, which models and modularize high-level reasoning and decision-making through key causal variables (value, belief, desire, and intention); $2)$ an action execution module, which translates internal intentions into embodied movements, enabling physically grounded interactions; and $3)$ an exploration module, which empowers the agent to actively explore the environment and update its mental states through feedback of actions. Our approach allows virtual humans to continuously adapt, learn, and evolve their behavior in response to environmental changes with autonomy, supporting dynamic and natural human-like interactions in the long horizon. Extensive experiments demonstrate the flexibility and scalability of our method in simulating individualized, daily-level behaviors in unknown environments.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 11069