Semantic Coherence Guided Multiview Similarity for Image Classification With Varied Supervision

Chunjie Zhang, Ping Cao, Huihui Bai, Xiaolong Zheng

Abstract: Pairwise similarity has been widely used for image classification by propagating the class information from labeled images to unlabeled images and predicting the classes of unlabeled images accordingly. Although widely used, pairwise similarity based classification methods have two drawbacks. On one hand, visual-semantic similarity consistency does not always hold due to semantic gap. On the other hand, the reliability of pairwise similarity is also influenced by the number and underlying distribution of labeled images. A well-designed classification model cannot be easily generalized to few-shot classification tasks. To solve these two problems, in this article, we propose a semantic coherence guided multiview similarity (SCG-MV) for image classification method with varied supervision levels. We measure the semantic coherence between image pairs for classifier learning. This is achieved by first conducting line search along the line segment between each image pair, and then measure the semantics of each interval. The similarity of one image pair can then be obtained by aggregating the semantic coherence of all the intervals. In this way, we can model image similarities more finely and discriminatively. To make use of multiview information, we also use the semantic coherence guided image similarities to learn multiview classifiers. The proposed semantic coherence guided method can be combined with other similarity based methods for image classification with various levels of supervision. We conduct both few-shot and fully supervised classification experiments on four datasets with the results well prove the discriminative power and generalization ability of the proposed method.