Adversarial Contrastive Training in Parameter Space for Improved Text Classification

Hao Wu, Xiangfeng Luo, Jianqi Gao, Dian Huang

Abstract: Fine-tuning pre-trained language models (PLMs) for downstream tasks has achieved remarkable success in various natural language processing (NLP) applications. To further enhance the overall performance of PLMs across different NLP tasks, we propose a novel adversarial contrastive training (ACT) method that incorporates adversarial perturbations in the parameter space. Specifically, ACT introduces adversarial perturbations to the model’s parameters, deliberately degrading its performance on downstream tasks. Subsequently, we apply contrastive learning to align the representations of specific tasks with those of the perturbed model, thereby improving the model’s robustness and generalization ability. Experimental results demonstrate that ACT significantly enhances the generalization performance of PLMs and achieves state-of-the-art results on several text classification benchmarks.

External IDs:dblp:conf/ijcnn/WuLGH25