Fault Tolerant Multi-Agent Learning with Adversarial Budget Constraints

David Henry Mguni, Yaqi Sun, Haojun Chen, Amir Darabi, Larry Olanrewaju Orimoloye, Yaodong Yang

Keywords: multi-agent reinforcement learning, robust, fault-tolerance, multi-agent systems

Abstract: Multi-agent reinforcement learning (MARL) often assumes that agents can reliably execute their intended actions. In practice, however, malfunctions and unexpected failures are inevitable, leading to severe coordination breakdowns. We propose the \emph{Multi-Agent Robust Training Algorithm (MARTA)}, a plug-and-play framework for training MARL policies that remain effective under agent failures. MARTA introduces a novel adversarial game in which a \emph{Switcher} learns when and where to activate malfunctions in high risk-states, while an \emph{Adversary} controls the faulty agents. The remaining agents are trained to \emph{jointly} best-respond to such targeted malfunctions, yielding policies that are robust to critical failures. We provide theoretical guarantees that MARTA converges to a Markov perfect equilibrium, ensuring robustness against worst-case malfunctions under both cost and budget formulations. Empirically, MARTA achieves state-of-the-art fault tolerance across diverse MARL benchmarks, including Traffic Junction, Level-Based Foraging, and Multi-Agent Particle Environments, substantially improving performance under both aligned and distribution-shifted failure scenarios. Our results highlight MARTA as a principled and general approach for fault-tolerant MARL.

Primary Area: reinforcement learning

Submission Number: 7323