Learning Good Policies By Learning Good Perceptual Models

Yilun Du, Phillip Isola

Keywords: visual representation learning, reinforcement learning, curiosity

TL;DR: We present a formulation of curiosity as a visual representation learning problem and show that it allows good visual representations in agents.

Abstract: Reinforcement learning (RL) has led to increasingly complex looking behavior in recent years. However, such complexity can be misleading and hides over-fitting. We find that visual representations may be a useful metric of complexity, and both correlates well objective optimization and causally effects reward optimization. We then propose curious representation learning (CRL) which allows us to use better visual representation learning algorithms to correspondingly increase visual representation in policy through an intrinsic objective on both simulated environments and transfer to real images. Finally, we show better visual representations induced by CRL allows us to obtain better performance on Atari without any reward than other curiosity objectives.