HuWo: Building Physical Interaction World Models for Humanoid Robot Locomotion
Keywords: Reinforcement Learning, Humanoid Locomotion, World Model, Sim-to-Real Transfer
Abstract: Reinforcement learning has proven effective for humanoid robot locomotion, yet achieving stable movement in complex environments remains challenging. Humanoid robots must maintain balance while navigating and continuously adapt to interactions with the environment. A deep understanding of these robot-environment dynamics is essential for achieving stable locomotion. Since there is privileged information that the robot cannot directly access, to expand the observable space, previous reinforcement learning-based methods either reconstruct environmental information from partial observations or reconstruct robotic dynamics information from partial observations, but they fall short of fully capturing the dynamics of robot-environment interactions. In this work, we propose an end-to-end reinforcement learning control framework based on physical interaction World Model for Humanoid Robots (HuWo). Our primary innovation is the introduction of a physical interaction world model to understand the dynamic interactions between the robot and the environment. Additionally, to address the temporal and dynamic nature of these interactions, we employ the hidden layers of Transformer-XL for implicit modeling. The proposed framework can showcase robust and flexible locomotion ability in complex environments such as slopes, stairs, and discontinuous surfaces. We validated the robustness of this method using the Zerith1 robot, both in simulations and real-world deployments, and quantitatively compared our HuWo against the baselines with better traversability and command-tracking.
Submission Number: 66
Loading