A Competition Winning Deep Reinforcement Learning Agent in microRTS

Scott Goodfriend

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: reinforcement learning, microRTS, PPO, RTS, imitation learning, behavior cloning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: This paper describes the first deep reinforcement learning agent to win the IEEE microRTS competition.

Abstract:

Scripted agents have predominantly won the five
previous iterations of the IEEE microRTS ($\mu$RTS) competitions hosted at CIG and
CoG. Despite Deep Reinforcement Learning (DRL) algorithms making significant strides
in real-time strategy (RTS) games, their adoption in this primarily academic
competition has been limited due to the considerable training resources required and the complexity
inherent in creating and debugging such agents. \agentName\ is the first DRL agent
to win the IEEE microRTS competition. In a benchmark without performance
constraints, \agentName\ regularly defeated the two
prior competition winners. This first competition-winning DRL submission can be
a benchmark for future microRTS competitions and a starting point for future DRL
research. Iteratively fine-tuning the base policy and transfer learning to specific maps were
critical to \agentName's winning performance. These strategies can be used in
economically training future DRL agents. Further work in Imitation Learning using Behavior Cloning and
fine-tuning these models with DRL has proven promising as an efficient way
to bootstrap models with novel behaviors.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6419