CHASM: Cooperative MARL-Based Approach for Efficient V2V Communication in CDSs

Bingyi Liu, Haoxiang Zhao, Enshu Wang, Weizhen Han, Zhuangzhuang Zhang, Libing Wu

Abstract: A group of Connected and Autonomous Vehicles (CAVs) with common interests can drive cooperatively, known as cooperative driving, and Cooperative Driving Systems (CDSs) for CAV s emerge as an effective approach to improving travel safety, requiring an efficient vehicle-to-vehicle communication scheme to periodically broadcast their kinetic status, i.e., beacon. However, current studies overlook the effects of hybrid transportation scenarios and intricate communication environments, such as those with limited communication resources that cause serious packet transmission collisions among vehicles, severely deteriorating communication performance and significantly compromising the stability of cooperative driving. To address the problem in intricate communication environments with limited communication resources, we regard each cooperative driving as an agent and formulate the decision-making process of cooperative driving as a Markov game. We solve this game via our proposed Cooperative Hierarchical Attention reinforcement learning framework in the Sequence Modelling manner (CHASM) consisting of manager and worker modules. Specifically, the manager module is equipped with graph attention networks to incorporate mutual influence among managers and operate at a lower temporal resolution to set goals for the worker module to overcome the delay reward challenge caused by transmission collisions. In the worker module, we design attention-based Sequential Decision Networks (SDNs) that cast cooperative reinforcement learning into the sequence modelling problem. We integrate the output of SDNs with the goal set by the manager module to generate farsighted and cooperative actions to enable agents to cooperatively use limited communication resources. We establish a simulator and perform extensive experiments to validate the effectiveness of CHASM.