PabLO: Improving Semi-Supervised Learning with Pseudolabeling Optimization

Harit Vishwakarma, Yi Chen, Satya Sai Srinath Namburi GNVV, Sui Jiet Tay, Ramya Korlakai Vinayak, Frederic Sala

Keywords: Semi-supervised Learning, Pseudo-Labeling, Self-Training, Confidence Functions

TL;DR: We introduce a framework that learns confidence functions and thresholds for semi-supervised learning (SSL), boosting test accuracy by up to 11% and reducing training iterations compared to standard methods.

Abstract: Modern semi-supervised learning (SSL) methods frequently rely on pseudolabeling and consistency regularization. The main technical challenge in pseudolabeling is identifying the points that can reliably be labeled. To address this challenge, we propose a framework to learn confidence functions and thresholds explicitly aligned with the SSL task, obviating the need for manual designs. Our approach formulates an optimization problem over a flexible space of confidence functions and thresholds, allowing us to obtain optimal scoring functions---while remaining compatible with the most popular and performant SSL techniques today. Extensive empirical evaluation of our method shows up to 11\% improvement in test accuracy over the standard baselines while requiring substantially fewer training iterations.

Submission Number: 34