Pragmatically Learning from Pedagogical Demonstrations in Multi-Goal Environments

Hugo Caselles-Dupré, Olivier Sigaud, Mohamed CHETOUANI

Keywords: learning from demonstration, multi-goal environments, pedagogical teaching, pragmatic reasoning

TL;DR: Building on the pedagogy and pragmatism concepts from Developmental Psychology, we show how learning from demonstration can benefit from a Bayesian goal inference mechanism to reduce goal ambiguity and learn faster in multi-goal environments.

Abstract: Learning from demonstration methods usually leverage close to optimal demonstrations to accelerate training. By contrast, when demonstrating a task, human teachers deviate from optimal demonstrations and pedagogically modify their behavior by giving demonstrations that best disambiguate the goal they want to demonstrate. Analogously, human learners excel at pragmatically inferring the intent of the teacher, facilitating communication between the two agents. These mechanisms are critical in the few demonstrations regime, where inferring the goal is more difficult. In this paper, we implement pedagogy and pragmatism mechanisms by leveraging a Bayesian model of Goal Inference from demonstrations. We highlight the benefits of this model in multi-goal teacher-learner setups with two artificial agents that learn with goal-conditioned Reinforcement Learning. We show that combining BGI-agents (a pedagogical teacher and a pragmatic learner) results in faster learning and reduced goal ambiguity over standard learning from demonstrations, especially in the few demonstrations regime.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/pragmatically-learning-from-pedagogical/code)