Label-guided Data Augmentation for Prompt-based Few Shot Learners

Anonymous

Abstract: Recent advances on large pre-trained language models (PLMs) lead impressive gains on many natural language understanding (NLU) tasks with task-specific fine-tuning. However, direct fine-tuning PLMs heavily rely on large amount of labeled instances, which are expensive and time-consuming to obtain. Prompt-based tuning on PLMs has proven valuable for few shot tasks. Existing works studying prompt-based tuning for few-shot NLU mainly focus on deriving proper label words with a verbalizer or generating prompt templates for eliciting semantics from PLMs. In addition, conventional data augmentation methods can enrich training data for improving few-shot learning, while ignoring the label semantics. It is promising to leverage the rich label semantics in label words for data augmentation to facilitate prompt-based tuning for the downstream NLU tasks. However, the work on this is rather limited. Therefore, we study a new problem of data augmentation for prompt-based few shot learners. We propose a novel label-guided data augmentation method PromptDA which exploits the enriched label semantic information for data augmentation. Experimental results on several few shot text classification tasks show that our proposed framework achieves superior performance by effectively leveraging label semantics and data augmentation in language understanding.

Paper Type: long