Now I Remember! Episodic Memory For Reinforcement Learning

Ricky Loynd, Matthew Hausknecht, Lihong Li, Li Deng

Abstract: Humans rely on episodic memory constantly, in remembering the name of someone they met 10 minutes ago, the plot of a movie as it unfolds, or where they parked the car. Endowing reinforcement learning agents with episodic memory is a key step on the path toward replicating human-like general intelligence. We analyze why standard RL agents lack episodic memory today, and why existing RL tasks don't require it. We design a new form of external memory called Masked Experience Memory, or MEM, modeled after key features of human episodic memory. To evaluate episodic memory we define an RL task based on the common children's game of Concentration. We find that a MEM RL agent leverages episodic memory effectively to master Concentration, unlike the baseline agents we tested.

TL;DR: Implementing and evaluating episodic memory for RL.

Keywords: Reinforcement learning, Deep learning, Episodic memory