Logical Composition in Lifelong Reinforcement Learning

Geraud Nangue Tasse, Steven James, Benjamin Rosman

Student First Author: Yes

Keywords: Reinforcement Learning, Lifelong learning, Multi task learning, Transfer learning, Zero shot composition

Abstract: The ability to produce novel behaviours from existing skills is an important property of lifelong learning agents. We build on recent work which formalises a Boolean algebra over the space of tasks and value functions, and show how this can be leveraged to tackle the ifelong learning problem. We propose an algorithm that determines whether a new task can be immediately solved using an agent’s existing abilities, or whether the task should be learned from scratch. We verify our approach in the Four Rooms domain, where an agent learns a set of skills throughout its lifetime, and then composes them to solve a combinatorially large number of new tasks in a zeroshot manner.

TL;DR: We propose a framework for lifelong learning that leverages zero-shot logical composition to solve new tasks that are expressible as logical combinations of previously learned tasks.