Geometric Projection of Information Manifolds for Robust Decision-Making with LLMs in Adversarial Driving Environments

xu zl, Qianqi Zhang, Yuntao Zou

Keywords: Information manifolds, Large language models, Adversarial robustness, Autonomous driving, Feature disentanglement, Geometric projection, Perception uncertainty, Few-shot learning

Abstract: Perception uncertainty poses a critical challenge for autonomous driving systems (ADS), where small-probability anomalies can lead to catastrophic failures in decision-making. While existing approaches rely on redundant sensors or multi-modal fusion, they struggle with rare edge cases and require extensive datasets for training. We propose LLM-ADF, a Large Language Model-based Autonomous Driving Framework that leverages few-shot learning to enhance robustness against perceptual anomalies. Our key innovation lies in constructing a specialized autonomous driving space through information geometry-guided dimensionality reduction, decoupling high-dimensional text embeddings into driving-relevant features while preserving contextual reasoning capabilities. We introduce a manifold-based reasoning mechanism that connects the text space with the driving space, enabling LLMs to perform spatial-temporal inference even under corrupted inputs. The framework incorporates a self-correction database that enables continuous learning from historical anomalies, dynamically adjusting the manifold structure through Fisher information metrics. We construct an adversarial dataset with 2,730 anomalous frames simulating sensor failures and adversarial attacks. Experimental results on UniAD and ST-P3 benchmarks demonstrate that LLM-ADF achieves 24.93% average collision rate on UniAD, outperforming GPT-Driver by 22% under normal conditions and showing 14.9% degradation under anomalies compared to 17-21% for existing LLM-based methods. Our approach represents a paradigm shift towards few-shot learning in safety-critical autonomous systems, providing theoretical foundations and practical solutions for L4 autonomous driving deployment.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 6191