Geometry-semantic aware for monocular 3D Semantic Scene Completion

Zonghao Lu, Bing Cao, Shuyin Xia, Qinghua Hu

Abstract: Highlights•We propose the Proxy-embedding Parallel Multi-task Network (PPM-Net).•PPM-Net integrates the feature representation capabilities of 2D and 3D networks.•We perform depth estimation and semantic segmentation proxy tasks on the 2D image.•We propose parallel 2D and 3D decoders along with the DHPP module.•DHPP module aggregates contextual information from perspective view features and voxelized grids.•We introduce a local-to-global loss to enhance the accuracy of occupied voxels.•PPMNet outperforms previous methods across multiple categories in the SemanticKITTI and NYUv2.