Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels

Adam Wei, Abhinav Agarwal, Boyuan Chen, Rohan Bosworth, Nicholas Ezra Pfaff, Russ Tedrake

Keywords: cotraining, imitation learning, simulated data

TL;DR: This paper presents a thorough empirical analysis of the principles and underlying mechanisms of cotraining from both simulated and real-world data in robot imitation learning.

Abstract: Cotraining with demonstrations generated both in simulation and on real hardware has emerged as a promising recipe for scaling imitation learning in robotics. This work seeks to elucidate basic principles of this sim-and-real cotraining to inform simulation design, sim-and-real dataset creation, and policy training. Our experiments confirm that cotraining with simulated data can dramatically improve performance, especially when real data is limited. We show that performance gains scale with additional simulated data until a plateau; adding more real data can increase this performance ceiling. The results also suggest that physical domain gaps may be more impactful than visual fidelity for contact-rich tasks. Perhaps surprisingly, some visual gap appears to help cotraining. We investigate this nuance and other mechanisms that facilitate positive transfer between sim-and-real. Lastly, we provide ablations for 2 alternative cotraining formulations. Videos can be found here: https://sim-and-real-cotraining.github.io/.

Submission Number: 23