Using Thermal Vision for Extended VINS-Mono to Localize Vehicles in Large-Scale Outdoor Road Environments

Mengwen He, Ragunathan Raj Rajkumar

Abstract: A monocular VIO (Visual-Inertial Odometry) system provides a compact, low-cost, and easily-deployed configuration for relative localization. However, using thermal vision for VIO is much less studied than using a visible-spectrum camera. A thermal-vision camera works under all lighting conditions and has been used to detect pedestrians, cars and animals at nighttime to provide ADAS (Advanced Driver Assistance System) functions. Common problems in directly using thermal images in conventional VIO methods are: (1) lower signal-to-noise ratio and fewer reliable feature points in the texture-less thermal images, (2) periodic recalibration hampers thermal image capturing and feature-point tracking, and (3) the “jello” effect of the rolling shutter readout architecture is sensitive to aggressive vehicle maneuvers. Extended VINS-Mono, proposed in our previous work [1], aims at providing relative and absolute localization of a vehicle in large-scale outdoor road environments by introducing (1) absolute localization methods to enable VINS-Mono to output local and global state estimates simultaneously, (2) vehicle speed readings for fast (re-)initialization and reliable scale estimates, and (3) DNN-based object detection methods to remove nonstationary objects from the visible scene. In this paper, we show that Extended VINS-Mono can use thermal images to provide relative and absolute localization even when light conditions are very poor. We conducted several experiments on a 25 Km-trip journey through highways, tunnels, urban areas and suburban areas in Pittsburgh, USA during daytime and nighttime to evaluate the performance of Extended Thermal VINS-Mono, including (re-)initialization, accuracy, rate, and latency. Our evaluation confirms that using thermal vision for localization satisfies localization requirements in large-scale outdoor road environments when the visible-spectrum camera performs poorly.