A Deeper Look at Optimal Transport for Imitation Learning
Abstract: Optimal transport (OT) tools have shown early promise for imitation learning (IL) and enable a metric-aware alignment of the expert and agent's stationary distributions. Despite encouraging results, the use of OT for IL is still ad hoc and lacks a systematic treatment, which could guide future research. To gain an understanding of these inner workings, we perform theoretical and empirical analysis of existing OT-based methods for IL from a unified perspective. Based on our study we propose OTIL, a simple and modality-agnostic method for IL. Our algorithm demonstrates state-of-the-art performance on a wide range of environments that feature both continuous and discrete action spaces, as well as state and image observations. We make our experimentation code public.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Fabio_Viola1
Submission Number: 217
Loading