MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning

Bohan Li, Longxu Dou, Yutai Hou, Yunlong Feng, Honglin Mu, Enbo Wang, Qingfu Zhu, Qinghua Sun, Wanxiang Che

Abstract: Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cloze problems by combining original input with a predetermined template. This approach demonstrates its effectiveness, especially in few-shot learning scenarios, where the model is trained on a scarce amount of data. Despite its successes, the limited templates and text in few-shot prompt-based learning scenarios leave significant room for performance improvement. Moreover, existing methods sometimes resort to model ensembles, which, while effective, could potentially hamper model efficiency due to increased computational demands [1]. To address these issues, we introduce MixPro, an augmentation method designed to augment both the vanilla input text and the templates. We implement this through the token-level, the sentence-level, and the template-level Mixup strategies. We conduct experiments on five few-shot datasets, and the results show that our MixPro achieves an average performance improvement of 5.08% compared to the backbone model before augmentation. Moreover, it outperforms other augmentation baselines, demonstrating its superior effectiveness.

External IDs:dblp:journals/mlc/LiDHFMWZSC25