Deep Interactive Image Semantic and Instance Segmentation

Xuzijing Wu, Pengfei Yang, Wenteng Shao, Tao Wang, Quansen Sun

Abstract: Existing deep interactive segmentation methods only focus on the extraction of specific instance objects. When the target of interest to users is a category, existing methods have to perform more interactions. We take advantages of the relevance between semantic and instance segmentations, and give two definitions based on the interactive mode, i.e. interactive semantic segmentation and interactive instance segmentation, to distinguish the actual requirements of instance objects and semantic categories that users are interested in. Then, a dual-branch-based full convolutional network model is proposed to integrate the above two tasks (similar to a multi-task learning), in which the relevance between the interactive semantic and instance segmentations can be jointly explored along with a simultaneous output of potential interested semantic categories and instance objects. Vast experimental results on GrabCut, Berkeley, SBD and DAVIS datasets verified the effectiveness of the proposed method.