Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Keywords: imitation learning, manipulation, sim-to-real
TL;DR: We propose a data generation system for automatically synthesizing corrective interventional data from a small set of human interventions.
Abstract: Imitation learning is a promising paradigm for training robot control policies, but these policies can suffer from distribution shift, where the conditions at evaluation time differ from those in the training data. One common real-world source of distribution shift is object pose estimation error, which can cause agents that rely on pose information to fail catastrophically during deployment. A popular approach for increasing policy robustness to distribution shift is interactive imitation learning, in which a human operator provides corrective interventions during policy deployment. However, collecting a sufficient amount of interventions to cover the distribution of policy mistakes can be burdensome for human operators. We propose Interventional MimicGen (I-MG), a novel data generation system that can autonomously generate a large set of corrective interventions with rich coverage of the state space from a small number of human interventions. We apply I-MG to policies deployed under object pose estimation error and show that it can increase policy robustness by up to 39x with only 10 human interventions. Videos and more are available at https://sites.google.com/view/interventional-mimicgen.
Submission Number: 6
Loading