SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment

Published: 05 Sept 2024, Last Modified: 11 Oct 2024CoRL 2024EveryoneRevisionsBibTeXCC BY 4.0
Keywords: Imitation Learning, Manipulation, Planning
TL;DR: We propose a skill-based approach to automated data generation for imitation learning.
Abstract: Imitation learning from human demonstrations is an effective paradigm for robot manipulation, but acquiring large datasets is costly and resource-intensive, especially for long-horizon tasks. To address this issue, we propose SkillGen, an automated system for generating demonstration datasets from a few human demos. SkillGen segments human demos into manipulation skills, adapts these skills to new contexts, and stitches them together through free-space transit and transfer motion. We also propose a Hybrid Skill Policy (HSP) framework for learning skill initiation, control, and termination components from SkillGen datasets, enabling skills to be sequenced using motion planning at test-time. We demonstrate that SkillGen greatly improves data generation and policy learning performance over a state-of-the-art data generation framework, resulting in the capability produce data for large scene variations, including clutter, and agents that are on average 24% more successful. We demonstrate the efficacy of SkillGen by generating over 24K demonstrations across 18 task variants in simulation from just 60 human demonstrations, and training proficient, often near-perfect, HSP agents. Finally, we apply SkillGen to 3 real-world manipulation tasks and demonstrate zero-shot sim-to-real transfer on a long-horizon assembly task. Videos, and more at https://skillgen.github.io.
Supplementary Material: zip
Website: https://skillgen.github.io/
Publication Agreement: pdf
Student Paper: no
Submission Number: 271
Loading