Adversarial Robustness of Multi-agent Reinforcement Learning Secondary Control of Islanded Inverter-based AC Microgrids

Ezzeldin Shereen, Kiarash Kazari, György Dán

Abstract: Secondary control of voltage magnitude and frequency is essential to the stable and secure operation of microgrids (MGs). Recent years have witnessed an increasing interest in developing secondary controllers based on multi-agent reinforcement learning (MARL), in order to replace existing model-based controllers. Nonetheless, unlike the vulnerabilities of model-based controllers, the vulnerability of MARLbased MG secondary controllers has so far not been addressed. In this paper, we investigate the vulnerability of MARL controllers to false data injection attacks (FDIAs). Based on a formulation of MG secondary control as a partially observable stochastic game (POSG), we propose to formulate the problem of computing FDIAs as a partially observable Markov decision process (POMDP), and we use state-of-the-art RL algorithms for solving the resulting problem. Based on extensive simulations of a MG with 4 distributed generators (DGs), our results show that MARL-based secondary controllers are more resilient to FDIAs compared to state of the art model-based controllers, both in terms of attack impact and in terms of the effort needed for computing impactful attacks. Our results can serve as additional arguments for employing MARL in future MG control.