Towards concurrent real-time audio-aware agents with deep reinforcement learning

Anton Debner, Vesa Hirvisalo

Keywords: audio, game engine, deep reinforcement learning, unity

TL;DR: We propose and evaluate a method for creating multiple concurrent audio-aware DRL agents with game engines, which are typically limited to a single audio listener.

Abstract:

Audio holds significant amount of information about our surroundings. It can be used to navigate, assess threats, communicate, as a source of curiosity, and to separate the sources of different sounds. Still, these rich properties of audio are not fully utilized by current video game agents.

We use spatial audio libraries in combination with deep reinforcement learning to allow agents to observe their surroundings and to navigate in their environment using audio cues. In general, game engines support rendering audio for one agent only. Using a hide-and-seek scenario in our experimentation we show how support for multiple concurrent listeners can be used to parallelize the runtime operation and to enable using multiple agents. Further, we analyze the effects of audio environment complexity to demonstrate the scalability of our approach.

Submission Number: 32