Help Me Explore: Combining Autotelic and Social Learning via Active Goal Queries

Ahmed Akakzia, Hugo Caselles-Dupré, Olivier Serris, Mohamed CHETOUANI, Olivier Sigaud, Cédric Colas

Keywords: Exploration, Teachability, Autotelism, Social Interaction, Curriculum Learning

TL;DR: We propose to blend individual and socially-guided skill learning in multi-goal environments with a new interaction protocol named Help Me Explore in which the agent chooses to seek external help for exploration or autonomously learn to master goals.

Abstract:

Most approaches to open-ended skill learning train a single agent in a purely sensorimotor environment. But because no human child learns everything on their own, we argue that sociality will be a key component of open-ended learning systems. This paper enables learning agents to blend individual and socially-guided skill learning through a new interaction protocol named Help Me Explore (HME).
In social episodes triggered at the agent's demand, a social partner suggests a goal at the frontier of the agent's capabilities and, when the goal is reached, follows up with a new adjacent goal just beyond. In individual episodes, the agent practices skills autonomously by pursuing goals it already discovered through either its own experience or social suggestions. The idea of augmenting an individual goal exploration with social goal suggestions is simple, general and powerful. We demonstrate its efficiency on two notoriously hard exploration benchmarks: continuous mazes and a 5-block robotic manipulation task. With minimal social interventions, an HME-agent outperforms the purely social agent deprived of its autonomy, and the purely individual agent which fails to solve hard exploration problems.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: General Machine Learning (ie none of the above)