Location Learning for AVs: LiDAR and Image Landmarks Fusion Localization with Graph Neural Networks

Qiyu Kang, Rui She, Sijie Wang, Wee Peng Tay, Diego Navarro Navarro, Andreas Hartmannsgruber

Abstract: High-accuracy vehicular self-localization plays an important role in autonomous driving. In this paper, we investigate the problem of estimating an autonomous vehicle's location using computer vision and LiDAR information, with respect to (w.r.t.) a reference map composed of landmarks from the environment. The map is generated off-line using static roadside objects such as traffic signs, traffic lights and road-side poles, which are organized into a graph for calibration. We use deep learning techniques to perform automatic feature extraction from sensor measurements. Specifically, we use a Convolutional Neural Network (CNN) to extract features from RGB images captured by an on-vehicle camera and use a Graph Neural Network (GNN) to integrate measurements from LiDAR scans. The vehicle's location is estimated from a regression neural network by comparing the extracted features from the real-time measurements with the calibration landmark map. In our experiments, we perform evaluations using 2 datasets and demonstrate that our approach achieves the state-of-the-art localization accuracy.