Ergodic imitation: Learning from what to do and what not to do

Aleksandra Ola Kalinowska, Ahalya Prabhakar, Kathleen Fitzsimons, Todd Murphey

Abstract:

With growing access to versatile robotics, it is
beneficial for end users to be able to teach robots tasks without
needing to code a control policy. One possibility is to teach
the robot through successful task executions. However, nearoptimal demonstrations of a task can be difficult to provide and
even successful demonstrations can fail to capture task aspects
key to robust skill replication. Here, we propose a learning
from demonstration (LfD) approach that enables learning
of robust task definitions without the need for near-optimal
demonstrations. We present a novel algorithmic framework for
learning tasks based on the ergodic metric—a measure of information content in motion. Moreover, we make use of negative
demonstrations—demonstrations of what not to do—and show
that they can help compensate for imperfect demonstrations,
reduce the number of demonstrations needed, and highlight
crucial task elements improving robot performance. In a proofof-concept example of cart-pole inversion, we show that negative
demonstrations alone can be sufficient to successfully learn
and recreate a skill. Through a human subject study with
24 participants, we show that consistently more information
about a task can be captured from combined positive and
negative (posneg) demonstrations than from the same amount
of just positive demonstrations. Finally, we demonstrate our
learning approach on simulated tasks of target reaching and
table cleaning with a 7-DoF Franka arm. Our results point
towards a future with robust, data-efficient LfD for novice users