Coarse-to-fine segmentation for indoor scenes with progressive supervision

Youcheng Song, Zhengxing Sun, Yunjie Wu, Hongyan Li

Abstract: Highlights • Segment 3D indoor scenes from coarse to fine imitating the way humans perceive. • Stacked network can produce segmentation results at different granularities. • Coarser-grained result provides context prior for finer-grained segmentation task. • Generate progressive supervision with hierarchical information for the training. Abstract Three-dimensional indoor scene segmentation is highly difficult due to the natural hierarchical structures and complicated contextual relationships in the scenes. In this paper, a 3D scene segmentation method that uses a stacked network is proposed for utilizing the context and hierarchy in 3D scenes. The method consists of two parts: a stacked network and progressive supervision. The stacked network consists of multiple base segmentation networks, and each network's output is concatenated to the raw input as another network's input to provide a prior context. Progressive supervision includes a group of coarse-to-fine segmentation labels that are generated based on the spatial relationships among objects in the scene, and it forces the network to learn the hierarchy. The experimental results from a regular dataset and a complex dataset demonstrate that our progressive supervision is effective and that our method outperforms existing methods in complex scenes.