MeUAL: Model-Enhanced Uncertainty-Aware Safe Reinforcement Learning for Safety-Critical Autonomous Highway Overtaking

Sunan Zhang, Bingbing Li, Boli Chen, Bo Hu, Chen Sun, Weichao Zhuang

Abstract: Decision-making and control are the core functionalities of high-level autonomous driving systems. Existing mainstream research, including modular and end-to-end paradigms, typically employ conservative strategies that compromise driving efficiency. However, driving efficiency constitutes a critical constraint on the transition of autonomous vehicles from mere operability to practical utility. Autonomous overtaking systems serve as a typical means to improve driving efficiency. Nevertheless, in stochastic and uncertain traffic scenarios, achieving safe and efficient continuous autonomous overtaking remains a significant challenge. In this context, this paper proposes a decision-making and control framework based on MeUAL to achieve the optimal trade-off between overtaking risk and efficiency. First, at the decision-making layer, a safe reinforcement learning method based on Uncertainty-aware Augmented Lagrangian (UAL) is developed to provide global overtaking guidance. Subsequently, the motion planning and control layer based on Model Predictive Control (MPC) closely tracks the UAL-generated guidance, while preserving the safety and constraint guarantees inherent to traditional MPC. Finally, a Policy Switching Mechanism (PSM) triggered by the safety epistemic uncertainty threshold is designed for the MeUAL-driven autonomous overtaking system. Experimental results demonstrate that MeUAL outperforms baseline algorithms with respect to reward-cost balance, sample efficiency, and learning stability. Moreover, in various test scenarios that are distinct from the training distribution, MeUAL-PSM exhibits strong robustness and interpretable overtaking maneuvers through flexible policy switching.

External IDs:dblp:journals/tits/ZhangLCHSZ26