Pseudo-Stereo++: Cycled Generative Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

Ahmed Elhagry, Hang Dai, Abdulmotaleb El-Saddik

Abstract: Recently, the feature-level generation has demonstrated the effectiveness of pseudo-stereo synthesis in Monocular 3D Detection (M3D). In this paper, we aim to further bridge the gap between the stereo and the monocular 3D object detectors in autonomous driving through direct image-level pseudo-stereo generation. We propose a novel Cycled Generative Pseudo-Stereo (CGPS) architecture to generate the right-view image from the left-view for constructing a pseudo-stereo pair to stereo 3D object detectors while maintaining the natural of M3D with the left-view image as the only input. Moreover, we use a triplet consistency loss to focus on the detected objects in the pseudo-stereo generation. Besides, we demonstrate that the proposed CGPS is an ad-hoc module to adapt top stereo 3D object detectors into monocular 3D object detectors. The proposed framework with CGPS achieves 74.80%, 55.28%, and 46.70% 3DAP for easy, moderate, and hard difficulty levels in monocular 3D detection on the KITTI benchmark with comparable performance to the stereo 3D object detectors but using a monocular image as the only input. Till the submission, the proposed M3D framework ranks 1 st with dramatic improvements against the existing monocular 3D detectors on the KITTI benchmark.