Lidar Waveforms are Worth 40x128x33 Words

Dominik Scheuble, Hanno Holzhüter, Steven Peters, Mario Bijelic, Felix Heide

Abstract: Lidar has become crucial for autonomous driving, providing high-resolution 3D scans that are key for accurate scene understanding. As shown on the right, lidar sensors measure the time-resolved full waveforms from the returning laser light, which a subsequent digital signal processor (DSP) converts to point clouds by identifying peaks in the waveform. Conventional automotive lidar DSP pipelines process each waveform individually, ignoring potentially valuable context from neighboring waveforms. As a result, lidar point clouds are prone to artifacts from low signal-to-noise ratio (SNR) regions, highly reflective objects, and environmental conditions like fog. In this work, we propose a neural DSP that jointly processes full waveforms using a transformer architecture leveraging features from adjacent waveforms to generate high-fidelity multi-echo point clouds. Trained on synthetic and real data, the method improves Chamfer distance by 32cm and 20cm compared to conventional peak finding and existing transient imaging approaches, respectively. This translates to maximum range improvements of up to 17m in fog and 14m in challenging real-world conditions