A Simple Semi-Supervised Joint Learning Framework for Few-shot Text Classification

Shaoshuai Lu, Long Chen, Wenjing Wang, Cai Xu, Wei Zhao, Ziyu Guan, Guangyue Lu

Abstract: The lack of labeled data is the bottleneck restricting deep text classification algorithm. State-of-the-art for most existing deep text classification methods follow the two-step transfer learning paradigm: pre-training a large model on an auxiliary task, and then fine-tuning the model on a labeled data. Their shortcoming is the high cost of training. To reduce training costs as well as alleviate the need for labeled data, we present a novel simple Semi-Supervised Joint Learning (SSJL) framework for few-shot text classification that captures the rich text semantics from large user-tagged data (referred to as weakly-labeled data) with noisy labels while also learning correct category distributions in small labeled data. We refine the contrastive loss function to better exploit inter-class contrastive patterns, making contrastive learning more applicable to the weakly-labeled setting. Besides, an appropriate temperature hyper-parameter can improve model robustness under label noise. The experimental results on four real-world datasets show that our approach outperforms the other baseline methods. Moreover, SSJL significantly boosts the deep models’ performance with only 0.5% (i.e. 32 samples) of the labeled data, showing its robustness in the data sparsity scenario.