Option Boosting
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Hierarchical Reinforcement Learning, Multi-Task Reinforcement Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: Constructing libraries of simple options which can be combined through an approach similar in flavour to boosting
Abstract: We introduce a novel approach to enhance stability and knowledge transfer in multi-task hierarchical reinforcement learning, specifically within the options framework. Modern Hierarchical Reinforcement Learning (HRL) algorithms can be prone to instability, due to the multilevel nature of the optimization process. To improve stability, we draw inspiration from boosting methods in supervised learning and propose a method which progressively introduces new options, while older options are kept fixed. In order to encourage generalization, each option policy has limited expressiveness. In order to improve knowledge transfer, we introduce the \textit{Option Library}, a mechanism to share options across a population of agents. Our approach improves learning stability and allows agents to leverage knowledge from simple tasks in order to explore and perform more complex tasks. We evaluate our algorithm in MiniGrid and CraftingWorld, two pixel-based 2D grid-world environments designed for goal-oriented tasks, which allows compositional solutions.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6049
Loading