Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching

Zhen Wu, Xiaoyu Huang, Lujie Yang, Yuanhang Zhang, Xi Chen, Pieter Abbeel, Rocky Duan, Angjoo Kanazawa, Carmelo Sferrazza, Guanya Shi, Karen Liu

Keywords: Robot Learning, Imitation Learning, Reinforcement Learning, Perception, Humanoids, Locomotion

TL;DR: We present Perceptive Humanoid Parkour, enabling agile, highly dynamic parkour by composing scarce human skills with locomotion via motion matching and distilling them into a single depth-based policy for long-horizon, contact-rich behaviors.

Abstract: We present Perceptive Humanoid Parkour (\method), a modular framework for autonomous, vision-based humanoid parkour. Our approach uses motion matching to compose retargeted atomic human skills into long-horizon kinematic trajectories with smooth transitions and natural dynamics. We then train motion-tracking RL experts and distill them into a single depth-based multi-skill policy via DAgger and RL. Using only onboard depth sensing and a discrete 2D velocity command, the robot autonomously selects and executes skills such as stepping over, climbing, vaulting, and rolling off obstacles of varying geometries and heights. Real-world experiments on a Unitree G1 humanoid demonstrate highly dynamic parkour behaviors, including climbing obstacles up to 1.25,m and long-horizon multi-obstacle traversal with closed-loop adaptation to obstacle perturbations.

Submission Number: 10