MODUS: An Impact-Aware Decision Framework with Adaptive Fusion for Connected Autonomous Vehicles

Quanlin Yu, Yuyang Xia, Shuncheng Liu, Weijie Lian, Rui Hu, Zhengzhuo Zhang, Shaozhi Wu, Kai Zheng, Han Su

Abstract: Autonomous driving is an emerging technology that has developed rapidly over the last decade. Benefiting from information sharing and collaborative decision-making of connected autonomous vehicles (CAVs), many researchers have started to design decision-making frameworks based on multi-agent reinforcement learning (MARL). However, existing methods primarily focus on how CAVs respond to the surrounding traffic but ignore their negative impacts on human-driven vehicles (HDVs). Furthermore, they always experiment in simple scenarios, only considering vehicle dynamics but not changing road structures and traffic light information. To address these limitations, we propose a collaborative decision framework based on MARL, called MODUS. Firstly, we propose a graph-based model to fuse multi-modal traffic information, which utilizes an encoder to exploit road semantic features and the attention mechanism to aggregate vehicle features adaptively. Secondly, to guide decision optimization, we design a hybrid reward function that incorporates a self-centered reward to optimize the driving performance of CAVs and a social impact reward to constrain their negative impacts on HDVs. Finally, a novel multi-agent reinforcement learning paradigm is proposed to train an optimal action policy for making collaborative decisions. Comprehensive experiments on a high-fidelity simulator Carla show that MODUS can advance state-of-the-art from multiple metrics.