Detecting Temporal Misalignment Attacks in Multimodal Fusion for Autonomous Driving

Md Hasan Shahriar, Md Mohaimin Al Barat, Harshavardhan Sundar, Ning Zhang, Naren Ramakrishnan, Thomas Hou, Wenjing Lou

Keywords: Multimodal Fusion, Temporal Misalignment Attack, Attack Detection, Autonomous Driving

TL;DR: We propose AION, a plug-in defense for autonomous driving that combines Continuity-Aware Contrastive Learning and DTW-based anomaly scoring to detect subtle temporal misalignments in multimodal sensor data.

Abstract:

Multimodal fusion (MMF) is crucial for autonomous driving perception, combining camera and LiDAR streams for reliable scene understanding. However, its reliance on precise temporal synchronization introduces a vulnerability: adversaries can exploit network-induced delays to subtly misalign sensor streams, degrading MMF performance. To address this, we propose AION, a lightweight, plug-in defense tailored for the autonomous driving scenario. AION integrates continuity-aware contrastive learning to learn smooth multimodal representations and a DTW-based detection mechanism to trace temporal alignment paths and generate misalignment scores.
AION demonstrates strong and consistent robustness against a wide range of temporal misalignment attacks on KITTI and nuScenes, achieving high average AUROC for camera-only (0.9493) and LiDAR-only (0.9495) attacks, while sustaining robust performance under joint cross-modal attacks (0.9195 on most attacks) with low false-positive rates across fusion backbones. Code is available at: https://github.com/shahriar0651/AION.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 23112