Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning

Anonymous

Keywords: Pre-training, Sample Efficiency

TL;DR: We show that pre-training encoders with a set of self-supervised learning tasks greatly improves performance in data-efficient RL.

Abstract:

Data efficiency poses a major challenge for deep reinforcement learning.
We approach this issue from the perspective of self-supervised representation learning, leveraging reward-free exploratory data to pretrain encoder networks.
We employ a novel combination of latent dynamics modelling and goal-reaching objectives, which exploit the inherent structure of data in reinforcement learning.
We demonstrate that our method scales well with network capacity and pretraining data.
When evaluated on the Atari 100k data-efficiency benchmark, our approach significantly outperforms previous methods combining unsupervised pretraining with task-specific finetuning, and approaches human-level performance.