Batch Differentiable Pose Refinement for In-The-Wild Camera/LiDAR Extrinsic Calibration

Lanke Frank Tarimo Fu, Maurice Fallon

Keywords: Sensor Fusion, Extrinsic Calibration, Differentiable Optimization

TL;DR: Differentiable pose refinement in batch achieves state-of-the-art zero-shot transfer for camera/LiDAR extrinsic calibration

Abstract: Accurate camera to LiDAR (Light Detection and Ranging) extrinsic calibration is important for robotic tasks carrying out tight sensor fusion --- such as target tracking and odometry. Calibration is typically performed before deployment in controlled conditions using calibration targets, however, this limits scalability and subsequent recalibration. We propose a novel approach for target-free camera-LiDAR calibration using end-to-end direct alignment which doesn't need calibration targets. Our batched formulation enhances sample efficiency during training and robustness at inference time. We present experimental results, on publicly available real-world data, demonstrating 1.6cm/0.07° median accuracy when transferred to unseen sensors from held-out data sequences. We also show state-of-the-art zero-shot transfer to unseen cameras, LiDARs, and environments.

Student First Author: yes

Instructions: I have read the instructions for authors (https://corl2023.org/instructions-for-authors/)