Semi-supervised Learning with Nearest-Neighbor Label and Consistency Regularization

Guolin Zheng, Zuoyong Li, Wenkai Hu, Haoyi Fan, Fum Yew Ching, Zhaochai Yu, Kaizhi Chen

Abstract: Semi-supervised learning, a system dedicated to making networks less dependent on labeled data, has become a popular paradigm due to its strong performance. A common approach is to use pseudo-labels with unlabeled data for training, however, pseudo-labels cannot correct their own errors. In this paper, we propose a semi-supervised method that uses nearest neighbor samples to obtain pseudo-labels and combines consistency regularization for image classification. Our method obtains pseudo-labels by computing the similarity of the data distribution between the weakly-augmented version of the unlabeled data and the labeled data stored in the support set and combines the consistency of the strongly-augmented version and the weakly-augmented version of the unlabeled data. We compared with several standard semi-supervised learning benchmarks and achieved a competitive performance. For example, we achieved an accuracy of \(94.02\%\) on CIFAR-10 with 250 labels and \(97.50\%\) on SVNH with 250 labels. It even achieved \(91.59\%\) accuracy with only 40 labels data in the CIFAR-10.