Multi-agent Deep FBSDE Representation For Large Scale Stochastic Differential Games

Tianrong Chen, Ziyi Wang, Ioannis Exarchos, Evangelos Theodorou

Keywords: Multi-agent Deep FBSDE Representation For Large Scale Stochastic Differential Games

Abstract: In this paper we present a deep learning framework for solving large-scale multi-agent non-cooperative stochastic games using fictitious play. The Hamilton-Jacobi-Bellman (HJB) PDE associated with each agent is reformulated into a set of Forward-Backward Stochastic Differential Equations (FBSDEs) and solved via forward sampling on a suitably defined neural network architecture. Decision-making in multi-agent systems suffers from the curse of dimensionality and strategy degeneration as the number of agents and time horizon increase. We propose a novel Deep FBSDE controller framework which is shown to outperform the current state-of-the-art deep fictitious play algorithm on a high dimensional inter-bank lending/borrowing problem. More importantly, our approach mitigates the curse of many agents and reduces computational and memory complexity, allowing us to scale up to 1,000 agents in simulation, a scale which, to the best of our knowledge, represents a new state of the art. Finally, we showcase the framework's applicability in robotics on a belief-space autonomous racing problem.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: In this paper, we propose a novel and scalable deep learning framework for solving multi-agent stochastic differential game using fictitious play.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=FCpUiXojTh