PA3DNet: 3-D Vehicle Detection With Pseudo Shape Segmentation and Adaptive Camera-LiDAR Fusion

Meiling Wang, Lin Zhao, Yufeng Yue

Abstract: 3-D vehicle detection is a key perception technique in autonomous driving. In this article, a novel 3-D vehicle detection framework that fuses camera images and Light Detection and Ranging (LiDAR) point clouds is proposed, named PA3DNet. The key novelties of PA3DNet are the proposing of a pseudo shape segmentation (PSS) model and an adaptive camera-LiDAR fusion (ACLF) module. The PSS model leverages self-assembled vehicle prototypes to learn shape-aware vehicle features. In order to achieve the adaptive fusion between visual semantics and LiDAR point features, learnable weight parameters are developed in the ACLF module to formulate an implicit complementarity between the two modalities. Extensive experiments on the widely used autonomous driving KITTI dataset demonstrate that PA3DNet achieves competitive accuracy when compared to advanced methods. It achieves 5.37% higher average precision (AP) on easy difficulty of 30–50 m and 9.67% higher AP on moderate difficulty of >50 m.