SELCOR: Self-Correction for Weakly Supervised Learning

Yang Zhou, Wee Sun Lee

Keywords: Weakly Supervised Learning, Label Noise, Self-Training

TL;DR: We propose a self-training based method to reduce noise in weak labels for weakly supervised learning.

Abstract: Powerful machine learning models often require training with large amounts of labeled data. Collecting precise labels from human supervision is expensive and time-consuming. Instead, it is much easier to obtain large-scale weak labels from multiple weak supervision sources such as rules or knowledge graphs, whereas weak labels could be noisy and make models prone to overfitting. We propose a self-training method for weakly supervised learning without using any true label. Our method learns a joint model with a corrector and a predictor, where the predictor generates pseudo clean labels and the corrector revises weak labels to reproduces the pseudo labels generated by the predictor. The joint model is trained by encouraging consistency between label generation and label correction, such that the predictor and corrector can iteratively improve each other to generate more reliable pseudo label for self-training. In this way, our method makes full use of weak labels and effectively suppresses label noise in weak labels and pseudo labels. Experiments on 8 benchmark datasets show that our method outperforms existing weakly supervised methods by large margins.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Unsupervised and Self-supervised learning