RankingMatch: Delving into Semi-Supervised Learning with Consistency Regularization and Ranking Loss

Trung Quang Tran, Mingu Kang, Daeyoung Kim

Keywords: BatchMean Triplet Loss, Semi-Supervised Learning, Consistency Regularization, Metric Learning

Abstract: Semi-supervised learning (SSL) has played an important role in leveraging unlabeled data when labeled data is limited. One of the most successful SSL approaches is based on consistency regularization, which encourages the model to produce unchanged with perturbed input. However, there has been less attention spent on inputs that have the same label. Motivated by the observation that the inputs having the same label should have the similar model outputs, we propose a novel method, RankingMatch, that considers not only the perturbed inputs but also the similarity among the inputs having the same label. We especially introduce a new objective function, dubbed BatchMean Triplet loss, which has the advantage of computational efficiency while taking into account all input samples. Our RankingMatch achieves state-of-the-art performance across many standard SSL benchmarks with a variety of labeled data amounts, including 95.13% accuracy on CIFAR-10 with 250 labels, 77.65% accuracy on CIFAR-100 with 10000 labels, 97.76% accuracy on SVHN with 250 labels, and 97.77% accuracy on SVHN with 1000 labels. We also perform an ablation study to prove the efficacy of the proposed BatchMean Triplet loss against existing versions of Triplet loss.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We propose RankingMatch, a novel semi-supervised learning method that encourages the model to produce the same prediction for not only the different augmented versions of the same input but also the samples from the same class.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=m04IexF8Yc