TaSIL: Taylor Series Imitation Learning

Daniel Pfrommer, Thomas TCK Zhang, Stephen Tu, Nikolai Matni

Keywords: learning theory, control theory, nonlinear systems, imitation learning

Abstract: We propose Taylor Series Imitation Learning (TaSIL), a simple augmentation to standard behavior cloning losses in the context of continuous control. TaSIL penalizes deviations in the higher-order Tayler series terms between the learned and expert policies. We show that experts satisfying a notion of incremental input-to-state stability are easy to learn, in the sense that that a small TaSIL-augmented imitation loss over expert trajectories guarantees a small imitation loss over trajectories generated by the learned policy. We provide sample-complexity bounds for TaSIL that scale as $\tilde{\mathcal{O}}(1/n)$ in the realizable setting, for $n$ the number of expert demonstrations. Finally, we demonstrate experimentally the relationship between the robustness of the expert policy and the order of Taylor expansion required in TaSIL, and compare standard Behavior Cloning, DART, and DAgger with TaSIL-loss-augmented variants. In all cases, we show significant improvement over baselines across a variety of MuJoCo tasks.

TL;DR: We propose Taylor Series Imitation Learning, a simple augmentation to standard behavior cloning losses that penalizes higher-order Tayler series terms of the error between the learned and expert policies.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2205.14812/code)