Robustness Evaluation of Multi-Agent Reinforcement Learning Algorithms using GNAs

Xusheng Zhang, Wei Zhang, Yishu Gong, Liangliang Yang, Jianyu Zhang, Zhengyu Chen, Sihong He

Keywords: multi-agent reinforcement learning, robustness

Abstract: Recently, multi-agent reinforcement learning (MARL) has shown its ability in solving sequential decision-making problems in complicated multi-agent environments. However, uncertainties from observations and executions undermine its performance when MARL methods are deployed in real-world applications. While crucial for deployment, a systematic robustness evaluation for MARL algorithms is not present. In this work, we utilize Gaussian noise attacks (GNAs) to examine the robustness of a benchmark MARL algorithm: multi-agent deep deterministic policy gradient (MADDPG). To the best of our knowledge, our work is the first to investigate the robustness of MADDPG to GNAs to observation and execution information. Our experiments show that GNA has totally different patterns in observation-wise attacks and execution-wise attacks. Furthermore, there are counter-intuitive insights from the experimental results which could guide researchers in future MARL methods development.