A Distributed Actor-Critic Algorithm and Applications to Mobile Sensor Network Coordination Problems

Paris Pennesi, Ioannis Ch. Paschalidis

Abstract: We introduce and establish the convergence of a distributed actor-critic method that orchestrates the coordination of multiple agents solving a general class of a Markov decision problem. The method leverages the centralized single-agent actor-critic algorithm of and uses a consensus-like algorithm for updating agents' policy parameters. As an application and to validate our approach we consider a reward collection problem as an instance of a multi-agent coordination problem in a partially known environment and subject to dynamical changes and communication constraints.