HoK3v3: an Environment for Generalization in Heterogeneous Multi-agent Reinforcement Learning

Lin Liu, Jianzhun Shao, Xinkai Chen, Yun Qu, Boyuan Wang, Zhenbin Ye, Yuexuan Tu, Hongyang Qin, Yang Jun Feng, Lin Lai, Yuanqin Wang, Meng Meng, Wenjun Wang, Xiyang Ji, QIANG FU, Lanxiao Huang, Minwen Deng, Yang Wei, Houqiang Li, Wengang Zhou et al. (7 additional authors not shown)

Keywords: Reinforcement learning, multi-agent reinforcement learning, competitive reinforcement learning

TL;DR: We introduce HoK3v3, a 3v3 game environment for multi-agent reinforcement learning (MARL) research, based on Honor of Kings, the world's most popular Multiplayer Online Battle Arena (MOBA) game at present.

Abstract: We introduce HoK3v3, a 3v3 game environment for multi-agent reinforcement learning (MARL) research, based on Honor of Kings, the world's most popular Multiplayer Online Battle Arena (MOBA) game at present. Due to the presence of diverse heroes and lineups (a.k.a., hero combinations), this environment poses a unique challenge for generalization in heterogeneous MARL. A detailed description of the tasks contained in HoK3v3, including observations, structured actions, and multi-head reward specifications, has been provided. We validate the environment by applying conventional MARL baseline algorithms. We examine the challenges of generalization through experiments involving the 3v3 MOBA full game task and its decomposed sub tasks, executed by lineups picked from the hero pool. The results indicate the limitations of existing RL methods in addressing scenarios that require heterogeneous generalization. All of the code, tutorial, encrypted game engine, can be accessed at: https://github.com/tencent-ailab/hok_env.

Submission Number: 118