Running Reinforcement Learning Agents on GPU for Many Simulations of Two-Person Simultaneous Games

Koichi Moriyama, Yoshiya Kurogi, Atsuko Mutoh, Tohgoroh Matsui, Nobuhiro Inuzuka

Abstract: It is desirable for multi-agent simulation to be run in parallel; if many agents run simultaneously, the total run time is reduced. It is popular to use GPGPU technology as an inexpensive parallelizing approach in simulation, but the “agents” runnable on GPU were simple, rule-based ones like elements in a scientific simulation. This work implements more complicated, learning agents on GPU. We consider an environment where many reinforcement learning agents learning their behavior in an iterated two-person simultaneous game while changing peers. It is necessary to run many simulations in each of which a pair of agents play the game. In this work, we implement on GPU the simulations where the agents learn with reinforcement learning and compare two methods assigning the simulations to GPU cores.