Multi-Resolution Skill Discovery for Hierarchical Reinforcement Learning

Published: 03 Nov 2023, Last Modified: 27 Nov 2023GCRL WorkshopEveryoneRevisionsBibTeX
Confirmation: I have read and confirm that at least one author will be attending the workshop in person if the submission is accepted
Keywords: Hierarchical reinforcement learning, Skill discovery, Offline reinforcement learning, Reinforcement learning
TL;DR: Multi-Resolution Skill Discovery
Abstract: Learning abstract actions can be beneficial for goal-conditioned reinforcement learning. Offline discovery of primitives has effectively leveraged large static datasets in reinforcement learning. While using abstract skills has performed well, the agents usually lack finesse in motion. Humans and animals, in contrast, can learn motor skills at different levels of temporal resolution, fine-grained skills such as piano playing, or gross skills such as running. We propose a solution to the problem of representing multiple temporal resolutions to enhance skill abstraction. We do so by encoding multiple temporal resolutions of skills and through an appropriate choice mechanism learned by an actor-critic framework. Our work builds on top of a recent work by Director and shows improved performance. We evaluate the method on the DeepMind control suite task 'walker_walk', resulting in qualitative and quantitative performance gains.
Supplementary Material: zip
Submission Number: 22