Explicit Recall for Efficient Exploration

Honghua Dong, Jiayuan Mao, Xinyue Cui, Lihong Li

Abstract: In this paper, we advocate the use of explicit memory for efficient exploration in reinforcement learning. This memory records structured trajectories that have led to interesting states in the past, and can be used by the agent to revisit those states more effectively. In high-dimensional decision making problems, where deep reinforcement learning is considered crucial, our approach provides a simple, transparent and effective way that can be naturally combined with complex, deep learning models. We show how such explicit memory may be used to enhance existing exploration algorithms such as intrinsically motivated ones and count-based ones, and demonstrate our method's advantages in various simulated environments.

Keywords: Exploration, goal-directed, deep reinforcement learning, explicit memory

TL;DR: We advocate the use of explicit memory for efficient exploration in reinforcement learning