Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification
Submission Type: Regular Long Paper
Submission Track: Machine Learning for NLP
Submission Track 2: Efficient Methods for NLP
Keywords: text classification, weak supervision, label noise, label bias
TL;DR: We propose simple methods to make weakly-supervised text classification based on seed-word matching achieve state-of-the-art.
Abstract: Recent advances in weakly supervised text classification mostly focus on designing sophisticated methods to turn high-level human heuristics into quality pseudo-labels. In this paper, we revisit the seed matching-based method, which is arguably the simplest way to generate pseudo-labels, and show that its power was greatly underestimated. We show that the limited performance of seed matching is largely due to the label bias injected by the simple seed-match rule, which prevents the classifier from learning reliable confidence for selecting high-quality pseudo-labels. Interestingly, simply deleting the seed words present in the matched input texts can mitigate the label bias and help learn better confidence. Subsequently, the performance achieved by seed matching can be improved significantly, making it on par with or even better than the state-of-the-art. Furthermore, to handle the case when the seed words are not made known, we propose to simply delete the word tokens in the input text randomly with a high deletion ratio. Remarkably, seed matching equipped with this random deletion method can often achieve even better performance than that with seed deletion.
Submission Number: 4574
Loading