Modeling Interaction-Aware Driving Behavior using Graph-Based Representations and Multi-Agent Reinforcement Learning

Fabian Konstantinidis, Moritz Sackmann, Ulrich Hofmann, Christoph Stiller

Abstract: Modeling the driving behavior of traffic partici-pants in highly interactive traffic situations, such as roundabouts, poses a significant challenge due to the complex interactions and the variety of traffic situations. To address this task, we propose a combination of graph-based representations of the environment with Multi-Agent Reinforcement Learning (MARL). By utilizing a graph-based representation of the local environment of each vehicle, our approach efficiently accounts for road structures and a varying number of surrounding vehicles interacting with each other. Building upon this representation, MARL enables us to learn a driving policy based on a minimal set of principles: drivers want to move along the road while avoiding collisions and maintaining comfortable accelerations. Sharing the learned policy among all agents allows us to leverage Proximal Policy Optimization (PPO), a policy gradient Reinforcement Learning (RL) algorithm. To evaluate our proposed model, we conduct experiments in a roundabout scenario from the INTERACTION dataset and compare it to a model learned via Behavior Cloning (BC). The results demonstrate that our proposed model is capable of maneuvering through dense traffic, indicating that our graph-based representation is well suitable for modeling and understanding complex road layouts and interactions between agents.