A‌ Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms

TMLR Paper6456 Authors

Abstract: The development of high-level autonomous driving (AD) is shifting from perception-centric limitations to a more fundamental bottleneck, i.e., a deficit in robust and generalizable reasoning. Although current systems manage structured environments, they consistently falter in long-tail scenarios and complex social interactions that require human-like judgment. Meanwhile, the advent of large language and multimodal models (LLMs and MLLMs) presents a transformative opportunity to integrate a powerful cognitive engine into AD systems, moving beyond pattern matching toward genuine comprehension. However, a systematic framework to guide this integration is critically lacking. To bridge this gap, we provide a comprehensive review of this emerging field, arguing that reasoning must be elevated from a modular component to the central cognitive core. Specifically, we first propose a novel Cognitive Hierarchy to deconstruct the monolithic driving task based on its cognitive and interactive complexity. Based on that, we further derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social game. Furthermore, we conduct a dual-perspective review of the state-of-the-art, analyzing both system-centric approaches to architecting intelligent agents and evaluation-centric practices for their validation. Our analysis reveals a clear trend toward holistic and interpretable "glass-box" agents. In conclusion, we identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control. For future work, the primary objective is to bridge the symbolic-to-physical gap, including verifiable neuro-symbolic architectures, robust reasoning under uncertainty, and scalable models for implicit social negotiation.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Blake_Aaron_Richards1

Submission Number: 6456