Fast Imitation via Behavior Foundation Models

Matteo Pirotta, Andrea Tirinzoni, Ahmed Touati, Alessandro Lazaric, Yann Ollivier

Keywords: Imitation learning, reinforcement learning, foundation models

TL;DR: Pre-trained behavioral foundation models can solve a variety of imitation learning tasks with few demonstrations and without any additional learning.

Abstract:

Imitation learning (IL) aims at producing agents that can imitate any
behavior given a few expert demonstrations. Yet existing approaches require
many demonstrations and/or running (online or offline) reinforcement
learning (RL) algorithms for each new imitation task. Here we show that recent RL foundation models based on successor measures can
imitate any expert behavior almost instantly
with just a few demonstrations and no need for RL or fine-tuning, while
accommodating several IL
principles (behavioral cloning, feature matching, reward-based, and
goal-based reductions).
In our
experiments, imitation via RL foundation models matches, and often
surpasses, the performance of SOTA offline IL algorithms, and produces
imitation policies from new demonstrations within seconds instead of hours.

Submission Number: 25