RTONet: Real-Time Occupancy Network for Semantic Scene Completion

Quan Lai, Haifeng Zheng, Xinxin Feng, Mingkui Zheng, Huacong Chen, Wenqiang Chen

Abstract: The comprehension of 3D semantic scenes holds paramount significance in autonomous driving and robotics technology. Nevertheless, the simultaneous achievement of real-time processing and high precision in complex, expansive outdoor environments poses a formidable challenge. In response to this challenge, we propose a novel occupancy network named RTONet, which is built on a teacher-student model. To enhance the ability of the network to recognize various objects, the decoder incorporates dilated convolution layers with different receptive fields and utilizes a multi-path structure. Furthermore, we develop an automatic frame selection algorithm to augment the guidance capability of the teacher network. The proposed method outperforms the existing grid-based approaches in semantic completion (mIoU), and achieves the state-of-the-art performance in terms of real-time inference speed while exhibiting competitive performance in scene completion (IoU) on the SemanticKITTI benchmark.