MATE: Benchmarking Multi-Agent Reinforcement Learning in Distributed Target Coverage Control

Xuehai Pan, Mickel Liu, fangwei zhong, Yaodong Yang, Song-Chun Zhu, Yizhou Wang

Keywords: Multi-agent Reinforcement Learning, Multi-Camera System, Asymmetric Game, Multi-agent Cooperative-Competitive Game

TL;DR: A gamification of the multi-camera multi-target target coverage problem, and an all-in-one multi-agent reinforcement learning benchmark

Abstract: We introduce the Multi-Agent Tracking Environment (MATE), a novel multi-agent environment simulates the target coverage control problems in the real world. MATE hosts an asymmetric cooperative-competitive game consisting of two groups of learning agents--"cameras" and "targets"--with opposing interests. Specifically, "cameras", a group of directional sensors, are mandated to actively control the directional perception area to maximize the coverage rate of targets. On the other side, "targets" are mobile agents that aim to transport cargo between multiple randomly assigned warehouses while minimizing the exposure to the camera sensor networks. To showcase the practicality of MATE, we benchmark the multi-agent reinforcement learning (MARL) algorithms from different aspects, including cooperation, communication, scalability, robustness, and asymmetric self-play. We start by reporting results for cooperative tasks using MARL algorithms (MAPPO, IPPO, QMIX, MADDPG) and the results after augmenting with multi-agent communication protocols (TarMAC, I2C). We then evaluate the effectiveness of the popular self-play techniques (PSRO, fictitious self-play) in an asymmetric zero-sum competitive game. This process of co-evolution between cameras and targets helps to realize a less exploitable camera network. We also observe the emergence of different roles of the target agents while incorporating I2C into target-target communication. MATE is written purely in Python and integrated with OpenAI Gym API to enhance user-friendliness. Our project is released at https://github.com/UnrealTracking/mate.

URL: https://github.com/UnrealTracking/mate

Dataset Url:

Source Code: https://github.com/UnrealTracking/mate
Documentation: https://mate-gym.readthedocs.io

License: MIT License

Author Statement: Yes

Contribution Process Agreement: Yes

In Person Attendance: Yes