Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames

Ev Zisselman, Mirco Mutti, Shelly Francis-Meretzki, Elisei Shafer, Aviv Tamar

Track: Robotics

Keywords: Imitation Learning, Robotics, Sequential Decision-making, Generalization, Video Games

TL;DR: We propose to clone the behavior of ``blindfolded'' experts that are compelled to employ non-trivial exploration to solve the task, which leads to better generalization.

Abstract: Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This ``blindfolded'' expert is compelled to employ non-trivial *exploration* to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside a videogame from the Procgen benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with videos and code: https://sites.google.com/view/blindfoldedexperts/home.

Serve As Reviewer: ~Ev_Zisselman1, ~Mirco_Mutti1, ~Aviv_Tamar2

Submission Number: 84