Generalized Behavior Learning from Diverse Demonstrations

Varshith Sreeramdass, Rohan R Paleja, Letian Chen, Sanne van Waveren, Matthew Gombolay

Keywords: Multimodal Imitation Learning, Task Relevant Diversity, Generalization over Latent Preferences

TL;DR: We propose a multimodal imitation learning algorithm that utilizes a novel diversity formulation to achieve generalization over latent preferences of demonstrators.

Abstract: Learning robot control policies through Reinforcement Learning can be challenging due to the complexity of designing rewards, which often result in unexpected behaviors. Imitation Learning overcomes this issue by using demonstrations to create policies that mimic expert behaviors. However, experts often demonstrate varied approaches to tasks. Capturing this variability is crucial for understanding and adapting to diverse scenarios. Prior methods capture variability by optimizing for behavior diversity alongside imitation. Yet, naive formulations of diversity can result in meaningless representation of latent factors, hindering generalization to novel scenarios. We propose Guided Strategy Discovery (GSD), a novel regularization method that specifically promotes expert-specified, task-relevant diversity. In the recovery of unseen expert behaviors, GSD improves 11\% over the next best baseline across three continuous control tasks on average. Code is available online at https://github.com/CORE-Robotics-Lab/GSD.

Submission Number: 30