Shape-guided segmentation for fine-grained visual categorization

Ming Sun, Jufeng Yang, Bo Sun, Kai Wang

Abstract: In this paper, we propose a shape-guided segmentation algorithm for fine-grained visual classification(FGVC). First, edge information is extracted from the query image and compared with each sample of training set, which can help us retrieve a subset of candidate proposals. These proposals are used to learn prior shape knowledge by separately estimating the foreground probabilities of corresponding pixels in the query image. Then, a redefined energy function is introduced to translate the minimum of energy to a good segmentation, with which we can dynamically pick out the most preferable proposal. After that, we obtain the label map of the image at the pixel level. Finally, the high-quality segmentation is used to aid locating semantic parts. We fine-tune one global model and two part models on Caffe to extract deep features and use a learned SVM classifier for categorization. We test three aspects in our experiment, including foreground segmentation, part localization and final classification. The results show that our method outperforms the state-of-the-art approaches on the famous Caltech-UCSD Birds 200-2011 dataset.