DeepCrawl: Deep Reinforcement Learning for Turn-based Strategy Games

Alessandro Sestini, Alexander Kuhnle, Andrew D. Bagdanov

Keywords: Machine Learning, ICML, Deep Reinforcement Learning, Human-Like Behavior, Game AI

Abstract: In this paper we introduce DeepCrawl, a fully-playable Roguelike prototype for iOS and Android in which all agents are controlled by policy networks trained using Deep Reinforcement Learning (DRL). Our aim is to understand whether recent advances in DRL can be used to develop convincing behavioral models for non-player characters in videogames. We begin with an analysis of requirements that such an AI system should satisfy in order to be practically applicable in video game development, and identify the elements of the DRL model used in the DeepCrawl prototype. The successes and limitations of DeepCrawl are documented through a series of playability tests performed on the final game. We believe that the techniques we propose offer insight into innovative new avenues for the development of behaviors for non-player characters in video games as they offer the potential to overcome some critical issues with classical approaches.