Improving heuristic search for RTS-game unit micromanagement using reinforcement learning

Supaphon Kamon, Tung Duc Nguyen, Tomohiro Harada, Ruck Thawonmas, Ikuko Nishikawa

Abstract: This paper proposes a method that uses reinforcement learning to improve heuristic search for unit micromanagement in real-time strategy (RTS) games. In the RTS game, unit micromanagement describes the detailed control of units, or, in other words, how the player controls their units. It decides all the commands that the player gives to their units such as the position, movement, abilities. One of the most commonly used algorithms for unit micromanagement is heuristic search. Due to the fact that the RTS game has large number of states and large action space, the heuristic search algorithm has to rely on evaluation methods that only search with a certain limited depth. We therefore apply reinforcement learning to achieve an evaluation method with high accuracy.