[Proposal-ML] End-to-End Scene Augmentation for Robust Robot Manipulation Learning

Chengbo Yuan, Shaoting Zhu, Suraj Joshi

Keywords: Robot Manipulation, diffusion model, segmentation model, dataset augmentation

TL;DR: Augmenting robot manipulation datasets with background scene changes using diffusion models to improve policy generalization across tasks.

Abstract: This project aims to enhance robot learning through dataset augmentation using diffusion models, and address the limitation of scarce large-scale, diverse datasets in robotics. Our method involves background scene manipulation to generate new samples in existing datasets, aiming to improve the generalization of learned policies for various robot learning tasks. The pipeline includes training segmentation and generative models to change the background of images while keeping the robot arm and object parts consistent. A combination of real-world, synthetic, and human-centric datasets is used to train the models.

Submission Number: 37