CAM-SegNet: a context-aware dense material segmentation network for sparsely labelled datasets

Yuwen Heng, Yihong Wu, Srinandan Dasmahapatra, Hansung Kim

Published: 01 Jan 2022, Last Modified: 05 Nov 202517th International Conference on Computer Vision Theory and Applications, 6/02/22EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Contextual information reduces the uncertainty in the dense material segmentation task to improve segmentation quality. Typical contextual information includes object, place labels or extracted feature maps by a neural network. Existing methods typically adopt a pre-trained network to generate contextual feature maps without fine-tuning since dedicated material datasets do not contain contextual labels. As a consequence, these contextual features may not improve the material segmentation performance. In consideration of this problem, this paper proposes a hybrid network architecture, the CAM-SegNet, to learn from contextual and material features during training jointly without extra contextual labels. The utility of our CAM-SegNet is demonstrated by guiding the network to learn boundary-related contextual features with the help of a self-training approach. Experiments show that CAM-SegNet can recognise materials that have similar appearances, achieving an improvement of 3-20% on accuracy and 6-28% on Mean IoU.