High Precision Real World Perception for Autonomous Driving

Min Bai

Abstract: The captivating hopes for a future with autonomous vehicles promises to free us from one of the most laborious, monotonous, and dangerous of our daily tasks. This thesis presents several works that I have undertaken to improve the perception capabilities of autonomous vehicles, with a particular attention to high precision. The first part of this thesis describes a technique to exploit the rich, high resolution camera information and the unambiguous 3D measurements of LiDAR to detect lane boundaries in 3D in real time with centimeter-level accuracy. We show that our approach significantly outperforms the baselines, particularly at longer distances. Next, we leverage the implicit boundary parameterization used in the previous work in a multi-task neural network for panoptic segmentation in the image space, with the goal of categorizing each pixel in an image into various semantic classes and delineating different instances of countable objects. We compare with the state-of-the-art technique and show that our approach can produce more accurate object boundaries. Finally, we bring temporal information into the segmentation task. Here, we leverage the full history of observations and reasoning results encoded into a non-parametric memory structure and intelligently incorporate new information at each instant in time to perceive small, safety critical objects such as construction elements in 3D.