Sniffer Faster R-CNN: A Joint Camera-LiDAR Object Detection Framework with Proposal Refinement

Sudip Dhakal, Qi Chen, Deyuan Qu, Dominic Carrillo, Qing Yang, Song Fu

Abstract: In this paper we present Sniffer Faster R-CNN (SFR-CNN), a novel camera-LiDAR sensor fusion framework for fast and accurate object detection in autonomous driving scenarios. The proposed detection framework architecture uses both LiDAR point clouds and Camera RGB images to generate region proposals. Current implementation of the regional proposal network (RPN) requires the generation of a large number of region proposals, majority of which are unproductive. As such, we devise a novel proposal refinement algorithm, to jointly optimize and filter a number of proposals in RPN through the combined application of both sets of LiDAR and image-based proposals thereby accelerating the LiDAR-Camera fusion algorithm without sacrificing detection precision and accuracy. Our experiments show that number of proposals is a complementary factor in determining the computational overhead in a detection network. Our proposed architecture is shown to produce state of art results on the KITTI joint object detection benchmark with the comparison being based on the execution time. While maintaining efficient detection accuracy we decrease the computational overhead by more than 20 % on the KITTI dataset.