Positive-Unlabeled Learning with Uncertainty-aware Pseudo-label Selection

Emilio Dorigatti, Jann Goschenhofer, Benjamin Schubert, Mina Rezaei, Bernd Bischl

Abstract: Positive-unlabeled (PU) learning aims at learning a binary classifier from only positive and unlabeled training data. Recent approaches addressed this problem via cost-sensitive learning by developing unbiased loss functions, and their performance was later improved by iterative pseudo-labeling solutions. However, such two-step procedures are vulnerable to incorrectly estimated pseudo-labels, as errors are propagated in later iterations when a new model is trained on erroneous predictions. To prevent such confirmation bias, we propose PUUPL, a novel loss-agnostic training procedure for PU learning that incorporates epistemic uncertainty in pseudo-label selection. By using an ensemble of neural networks and assigning pseudo-labels based on low-uncertainty predictions, we show that PUUPL improves the reliability of pseudo-labels, increasing the predictive performance of our method and leading to new state-of-the-art results in self-training for PU learning. With extensive experiments, we show the effectiveness of our method over different datasets, modalities, and learning tasks, as well as improved calibration, robustness over prior misspecifications, biased positive data, and imbalanced datasets.