A Compression-Inspired Framework for Macro Discovery

Francisco M. Garcia, Bruno C. da Silva, Philip S. Thomas

Abstract: We consider the problem of how a reinforcement learning agent, tasked with solving a set of related Markov decision processes, can use knowledge acquired early on in its lifetime to improve its ability to more rapidly solve novel tasks. We propose a three-step framework that generates a diverse set of macros that lead to high rewards when solving a set of related tasks. Our experiments show that augmenting the original action-set of the agent with the identified macros allows it to more rapidly learn optimal policies in novel MDPs.