Large Batch Sharing

Thibault Lahire

Keywords: Reinforcement Learning, Multi-Agent Systems, Importance Sampling

Abstract:

By reusing experiences collected from past different policies, experience replay significantly improves the training efficiency of reinforcement learning algorithms.
Rapid convergence occurs when learning is based on pertinent experiences that offer valuable information.
Nonetheless, how to effectively combine experience replay with multi-agent reinforcement learning is still an open challenge.
We study how sharing collected experiences helps the training process and show that sharing a small amount of selected experiences between agents improves the learning process compared to the baseline where each agent is independent. The shared experiences are selected by each agent on internal statistics, ensuring their meaningfulness.
Our first results on the multi-agent Pursuit environment highlight an improvement by a substantial margin and need to be consolidated by complementary experiments.

Submission Number: 1