Multi-Resolution Skill Discovery for Hierarchical Reinforcement Learning

Shashank Sharma, Vinay Namboodiri, Janina Hoffmann

Confirmation: I have read and confirm that at least one author will be attending the workshop in person if the submission is accepted

Keywords: Hierarchical reinforcement learning, Skill discovery, Offline reinforcement learning, Reinforcement learning

TL;DR: Multi-Resolution Skill Discovery

Abstract:

Learning abstract actions can be beneficial for goal-conditioned reinforcement learning.
Offline discovery of primitives has effectively leveraged large static datasets in reinforcement learning.
While using abstract skills has performed well, the agents usually lack finesse in motion.
Humans and animals, in contrast, can learn motor skills at different levels of temporal resolution, fine-grained skills such as piano playing, or gross skills such as running.
We propose a solution to the problem of representing multiple temporal resolutions to enhance skill abstraction.
We do so by encoding multiple temporal resolutions of skills and through an appropriate choice mechanism learned by an actor-critic framework.
Our work builds on top of a recent work by Director and shows improved performance.
We evaluate the method on the DeepMind control suite task 'walker_walk', resulting in qualitative and quantitative performance gains.

Submission Number: 22