HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning and Adaptation

Kevin Lin, Ajay Mandlekar, Caelan Reed Garrett, Nikita Chernyadev, Yu Fang, Runyu Ding, Yuqi Xie, Linxi Fan, Yuke Zhu

Keywords: Imitation Learning, Robot Learning, Synthetic Data Generation

TL;DR: HumanoidMimicGen generates large-scale, humanoid loco-manipulation demonstrations by adapting whole-body skills and interleaving them with planning, enabling effective imitation learning in simulation and co-training for real-world deployment.

Abstract: Imitation learning is a promising approach for training humanoid robots to simultaneously walk and manipulate, but it requires a large number of demonstrations, which are time-intensive and difficult to collect via teleoperation. Existing data-generation algorithms can automatically synthesize demonstrations for manipulators. But they struggle to directly transfer to humanoids because their high-dimensional composite action spaces involve arms, legs, and torsos. We present HumanoidMimicGen, an algorithm for generating humanoid legged loco-manipulation data. Our algorithm adapts contact-rich whole-body skills from a handful of source demonstrations to new states, generalizing across changes in object pose. By interleaving these single- and dual-arm skills with whole-body locomotion and manipulation planning, the method generates stable, collision-free data across diverse scenes and layouts. To evaluate our approach, we developed a new simulated loco-manipulation benchmark. There, we demonstrate that HumanoidMimicGen automatically generates large datasets for imitation learning and enables a systematic study of how data generation and policy learning decisions impact model performance. We show that we can use a small amount of real-world data along with HumanoidMimicGen data to effectively train capable real-world whole-body visuomotor policies.

Submission Number: 30