Multi-Agent Reinforcement Learning with Epistemic Priors

Thayne T. Walker, Jaime S. Ide, Minkyu Choi, Michael John Guarino, Kevin Alcedo

Keywords: Multi-Agent, Reinforcement Learning, Epis

TL;DR: Estimating the states of agents we cannot observe in MARL helps a lot.

Abstract: It is important for autonomous multi-agent teams to coordinate actions so collaborative goals can be achieved efficiently without conflicts. Without coordination, the goal may be achieved inefficiently, or in the worst case, not at all. Practical issues in multi-agent, real-time systems are limited sensing and communication capabilities. A significant number of multi-agent algorithms rely on accurate state information for all agents in order to effectively coordinate. In this paper, we propose an approach called Reinforcement Learning with Epistemic Priors (MARL-EP). MARL-EP uses epistemic estimation of the knowledge and actions of other agents from planners to infer portions of the observation space which are unobservable. We show that MARL-EP allows a very high level of coordination to be achieved with severely impaired sensing and zero communication between agents.

Submission Number: 13