Deep Spectral Clustering for Object Instance Segmentation

Matt Barnes, Artur Dubrawski

Abstract: We introduce a new graph Laplacian which is only available during training time and show how existing feedforward networks can learn to predict its eigenvectors. This is particularly useful for the problem of image segmentation, as it allows converting existing semantic segmentation networks (a basic multi-class classification problem) into instance segmentation networks (a more complex structured prediction problem). We show some convenient theoretical properties of this graph Laplacian, propose several rounding schemes (with error bounds) of the eigenvectors, and present preliminary results showing an insignificant change in error after converting from semantic to instance segmentation on the PASCAL VOC dataset.

Keywords: Spectral clustering, image segmentation, deep learning, graph Laplacian

TL;DR: Optimize the spectral loss of a network to retrain a semantic segmentation network (e.g. FCN) for instance segmentation (a more complex structured prediction problem).