Unleashing the Power of Annotation: Enhancing Semi-Supervised Learning through Unsupervised Sample Selection

Qian Shao, Jiangrui Kang, Qiyuan Chen, Hongxia Xu, Yiwen Cao, JIAJUAN LIANG, Jian Wu

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: semi-supervised learning, unsupervised sample selection

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Proposed an unsupervised sample selection method to select data for annotation from the unlabeled data to improve the performance of semi-supervised learning methods.

Abstract: With large volumes of unlabeled data and limited annotation budgets, Semi-Supervised Learning (SSL) has become a preferred approach in many deep learning tasks. However, most previous studies have primarily focused on utilizing labeled and unlabeled data for model training to improve performance, while the efficient selection of samples for annotation under budgetary constraints has often been overlooked.To fill this gap, we propose an efficient sample selection methodology named Unleashing the Power of Annotation (UPA). By adopting a modified Frank-Wolfe algorithm to minimizing a novel criterion $\alpha$-Maximum Mean Discrepancy ($\alpha$-MMD), UPA selects a representative and diverse subset for annotation from the unlabeled data. Furthermore, we demonstrate that minimizing $\alpha$-MMD enhances the generalization ability of low-budget learning. Experiments show that UPA consistently improves the performance of several popular SSL methods, surpassing various prevailing Active Learning (AL) and Semi-Supervised Active Learning (SSAL) methods even under constrained annotation budgets.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 654