Improving Data Augmentation for Multi-Modality 3D Object Detection

Wenwei Zhang, Zhe Wang, Chen Change Loy

Abstract: Single-modality object detectors have witnessed a drastic boost in the past few years thanks to the well-explored data augmentation and training techniques. On the contrary, multi-modality detectors adopt relatively simple data augmentation due to difficulty in ensuring cross modality consistency between point clouds and images. Such a limitation hampers fusion effectiveness and performance growth of multi-modality detectors. Therefore, we contribute a pipeline, named transformation flow, to bridge the gap between single and multi-modality data augmentation with transformation reversing and replaying. In addition, considering occlusions, a point in different modalities may be occupied by different objects, making augmentations such as cut and paste non-trivial for multi-modality detection. We further present Multi-mOdality Cut and pAste (MoCa), which simultaneously considers occlusion and physical plausibility to maintain the multi-modality consistency. Without using ensemble of detectors, our multi-modality detector achieves new state-of-the-art performance on nuScenes dataset and competitive performance on KITTI 3D benchmark. Code and models will be released.

Track: Original Contribution

Type: PDF