LLM-Informed Semi-Supervised Learning for Text Classification
Keywords: semi-supervised learning, LLM
TL;DR: Fuse consistency learning and large language models to improve model capabilities in semi-supervised learning setups with scarce labels.
Abstract: Large Language Models (LLMs) have shown impressive zero-shot and few-shot capabilities in many NLP tasks including text classification. While these models outperform others in terms of raw performance when few examples are available, they are expensive to use in practice and may lag behind traditional approaches when data (labeled or unlabeled) is plentiful. Semi-supervised learning (SSL) on the other hand can utilize large amounts of unlabeled data in combination with labeled data to improve a model's performance. In this paper, we propose to unify LLM and SSL under a common framework which effectively leverages the few-shot capabilities of LLMs in combination with SSL's ability to extract valuable information from unlabeled data to improve the model capabilities in text classification. Our approach, entitled LLM-SSL, utilizes LLMs to generate predictions on unlabeled examples and uses these predictions to guide the SSL training and improve the quality of pseudo-labels during training. We show that LLM-SSL outperforms both prior SSL approaches as well as few-shot LLMs on six text classification benchmarks.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10280
Loading