Reset-Free Lifelong Learning with Skill-Space Planning

Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch

Keywords: reset-free, lifelong, reinforcement learning

Abstract: The objective of \textit{lifelong} reinforcement learning (RL) is to optimize agents which can continuously adapt and interact in changing environments. However, current RL approaches fail drastically when environments are non-stationary and interactions are non-episodic. We propose \textit{Lifelong Skill Planning} (LiSP), an algorithmic framework for lifelong RL based on planning in an abstract space of higher-order skills. We learn the skills in an unsupervised manner using intrinsic rewards and plan over the learned skills using a learned dynamics model. Moreover, our framework permits skill discovery even from offline data, thereby reducing the need for excessive real-world interactions. We demonstrate empirically that LiSP successfully enables long-horizon planning and learns agents that can avoid catastrophic failures even in challenging non-stationary and non-episodic environments derived from gridworld and MuJoCo benchmarks.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ kzl/lifelong_rl](https://github.com/kzl/lifelong_rl)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2012.03548/code)