Decrease the Prompt Uncertainty: Adversarial Prompt Learning for Few-shot Text Classification

Jinta Weng, Zhaoguang Zhang, Yaqi Jing, Chenxu Niu, Heyan Huang, Yue Hu

Abstract: With few-shot learning abilities, pre-trained language models (PLMs) have achieved remarkable success in classification tasks. However, recent studies have shown that the performance of PLM is vulnerable due to different prompts and the instability of the prompt-based learning process. To address this challenge, we explore appropriate perturbation addition of adversarial training and integrate the global knowledge of the full-parameter fine-tuned pre-trained language model(PLM). Specifically, we propose an adversarial prompt learning model (ATPET) and ATPET with fine-tuning(ATPET-FT), incorporating ATPET with fine-tuning knowledge into the prompt learning process. Through extensive experiments on several few-shot classification tasks and challenging data settings, we demonstrate that our methods consistently improve the robustness while maintaining the effectiveness of PLMs.

External IDs:dblp:conf/smc/WengZJNH024