Learning Task Agnostic Skills with Data-driven Guidance

Even Klemsdal, Sverre Herland, Abdulmajid Murad

Keywords: Artificial Intelligence, Unsupervised Reinforcement Learning, Reinforcement Learning, Skill discovery

TL;DR: Using expert data to narrow the search space for skill discovery algorithms.

Abstract: To increase autonomy in reinforcement learning, agents need to learn useful behaviours without reliance on manually designed reward functions. To that end, skill discovery methods have been used to learn the intrinsic options available to an agent using task-agnostic objectives. However, without the guidance of task-specific rewards, emergent behaviours are generally useless due to the under-constrained problem of skill discovery in complex and high-dimensional spaces. This paper proposes a framework for guiding the skill discovery towards the subset of expert-visited states using a learned state projection. We apply our method in various reinforcement learning (RL) tasks and show that such a projection results in more useful behaviours.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2108.01869/code)