Uncertainty-aware Panoptic Segmentation for Camera and LiDAR Data

Kshitij Sirohi, Mohammad Sajad Marvi, Wolfram Burgard

Keywords: Uncertainty Estimation, Panoptic Segmentation, LiDAR segmentation, Image Segmentation

TL;DR: This paper presents novel networks for the camera image and LiDAR scanners uncertainty-aware panoptic segmentation for robust holistic perception.

Abstract: Reliable scene understanding is indispensable for modern autonomous systems. Current learning-based methods typically try to maximize their performance based on segmentation metrics that only consider the quality of the segmentation. However, for the safe operation of a system in the real world it is crucial to consider the uncertainty in the prediction as well. In this work, we discuss the task of uncertainty-aware panoptic segmentation, which aims to predict per-pixel semantic and instance segmentations, together with per-pixel uncertainty estimates. We present two novel Evidential Panoptic Segmentation Networks, EvPSNet for solving this task with camera images, and EvLPSNet for LiDAR data. We provide several strong baselines combining state-of-the-art panoptic segmentation networks with sampling-free uncertainty estimation techniques for comparison. Extensive evaluations show that our approaches achieve the new state-of-the-art for the uncertainty-aware Panoptic Quality (uPQ) and the panoptic Expected Calibration Error (pECE). We make our code available at: https://github.com/kshitij3112