Distributed Deep Policy Gradient for Competitive Adversarial Environment

Denis Osipychev, Girish Chowdhary

Abstract: This work considers the problem of cooperative learners in partially observable, stochastic environment, receiving feedback in the form of joint reward. The paper presents a flexible multi-agent competitive environment for online training and direct policy performance comparison. This forms a formal problem of a multi-agent Reinforcement Learning (RL) under partial observability, where the goal is to maximize the score performance measured in a direct confrontation. To address the complexity of the problem we propose a distributed deep stochastic policy gradient with individual observations, experience replay, policy transfer, and self-play.

Keywords: multi-agent, partially observable, reinforcement learning, deepRL, self play, competitive environment