PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitation

Yunze Liu, Changxi Chen, Chenjing Ding, Li Yi

Abstract: Humanoid Reaction Synthesis is pivotal for creating highly interactive and empathetic robots that can seamlessly integrate into human environments, enhancing the way we live, work, and communicate. However, it is difficult to learn the diverse interaction patterns of multiple humans and generate physically plausible reactions. Currently, the predominant approaches involve kinematics-based and physics-based methods. The kinematic-based methods lack physical prior limiting their capacity to generate convincingly realistic motions. The physics-based method often relies on kinematics-based methods to generate reference states, which struggle with the challenges posed by kinematic noise during action execution. Moreover, these methods are unable to achieve real-time inference constrained by their reliance on diffusion models. In this work, we propose a Forward Dynamics Guided 4D Imitation method to generate physically plausible human-like reactions. The learned policy is capable of generating physically plausible and human-like reactions in real-time, significantly improving the speed(x33) for inference and quality of reactions compared with the existing methods. Our experiments on the InterHuman and Chi3D datasets, along with ablation studies, demonstrate the effectiveness of our approach. More visualizations are available in supplementary materials.

Primary Subject Area: [Generation] Generative Multimedia

Relevance To Conference: Humanoid Reaction Synthesis and Generative Multimedia are closely related in their aim to enhance multimedia experiences. Generative models like Diffusion models and Generative Adversarial Networks, central to Generative Multimedia, can produce highly realistic and diverse content. When applied to Humanoid Reaction Synthesis, these models enable the creation of virtual characters with lifelike reactions and expressions. This integration not only increases the realism of humanoid characters but also allows for personalized and interactive experiences, as the characters can adapt their responses based on user interactions. The synergy between these fields contributes to more immersive and engaging multimedia systems.

Submission Number: 1338