Robust Multi-Agent Reinforcement Learning for Autonomous Vehicle in Noisy Highway Environments

Lilan Lin, Xiaotong Nie, Jian Hou

Keywords: Autonomous driving, multi-agent deep reinforcement learning, adversarial learning, curriculum learning

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: We develop Robust MARL in Autonomous Vehicle Highway On-ramp Merging in noisy environments

Abstract: The field of research on multi-agent reinforcement learning (MARL) algorithms in self-driving vehicles is rapidly expanding in mixed-traffic scenarios where autonomous vehicles (AVs) and human-driven vehicles (HDVs) coexist. Most studies assume that all AVs can obtain accurate state information. However, in real-world scenarios, noisy sensor measurements have a significant impact. To address this issue, we propose an effective and robust MARL algorithm Multi-Agent Proximal Policy Optimization with Curriculum-based Adversarial Learning (CA-MAPPO) for situations where the observation perturbations are considered. The proposed approach incorporates adversarial samples during training and adopts a curriculum learning approach by gradually increasing the noise intensity. By evaluating the proposed approach in the ideal environment and scenarios under noise attacks with varying intensities, experiment results demonstrate that the proposed algorithm enables AVs to achieve a success rate of over 70\% for the multi-lane highway on-ramp merging task, achieving a maximum average speed of up to over 19 $m/s$ and performing significantly better than the state-of-the-art MARL algorithms such as MAPPO and MAACKTR.

Primary Area: Deep Learning (architectures, deep reinforcement learning, generative models, deep learning theory, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: Yes

Submission Number: 368