Learning to play visual doom using model-free episodic control

Byeong-Jun Min, Kyung-Joong Kim

Abstract: Recently, the deep reinforcement learning has shown successful outcomes in classic video games (e.g., ATARI) and visual doom competition. Although it's very powerful, it suffers from very long learning time to generalize its performance. For example, it takes about 7~15 days to produce a good controller for ATARI games with state-of-the art GPUs. In this work, we propose to speed up the visual-based learning by introducing episodic control into the Visual Doom platform. The episodic control memorizes agent's experience with random projection and selects the next action based on similarity search on the memory. Because it's a model-free learning, it does not require much time to generalize a model and speeds up learning by exploiting previous experience. This is the first time to apply the episodic control into the visual Doom platform. Experimental results show that it converges to the desirable performance faster than the deep Q network in basic environment.