Generalized Behavior Learning from Diverse Demonstrations

Published: 05 Nov 2023, Last Modified: 31 Oct 2023OOD Workshop @ CoRL 2023EveryoneRevisionsBibTeX
Keywords: Multimodal Imitation Learning, Task Relevant Diversity, Generalization over Latent Preferences
TL;DR: We propose a multimodal imitation learning algorithm that utilizes a novel diversity formulation to achieve generalization over latent preferences of demonstrators.
Abstract: Learning robot control policies through Reinforcement Learning can be challenging due to the complexity of designing rewards, which often result in unexpected behaviors. Imitation Learning overcomes this issue by using demonstrations to create policies that mimic expert behaviors. However, experts often demonstrate varied approaches to tasks. Capturing this variability is crucial for understanding and adapting to diverse scenarios. Prior methods capture variability by optimizing for behavior diversity alongside imitation. Yet, naive formulations of diversity can result in meaningless representation of latent factors, hindering generalization to novel scenarios. We propose Guided Strategy Discovery (GSD), a novel regularization method that specifically promotes expert-specified, task-relevant diversity. In the recovery of unseen expert behaviors, GSD improves 11\% over the next best baseline across three continuous control tasks on average. Code is available online at https://github.com/CORE-Robotics-Lab/GSD.
Submission Number: 30
Loading