Rethinking Domain Generalization for Text: Classifier-Only Training with Test-Time Domain Shift Correction

ACL ARR 2026 January Submission660 Authors

Keywords: Domain Generalization, Pretrained Language Model, Text Classification

Abstract: Domain generalization aims to enable models trained on source domains to generalize to previously unseen target domains. In text classification, most existing methods explicitly learn domain-invariant representations through adversarial or contrastive objectives on top of pretrained language models (PLMs). However, PLMs are already trained on massive corpora and inherently encode substantial domain-invariant semantics, making additional representation learning potentially unnecessary and even harmful due to source-domain overfitting. In this work, we rethink text domain generalization from a representation-preserving perspective and propose Classifier Only for Source (CO4S), a simple plug-in framework. CO4S fully freezes the pretrained backbone and trains only a lightweight classifier on the source domain(s). At test time, it estimates a domain shift vector from the difference between the mean sentence representations of the source and target domains, and applies this shift to align target representations before classification. This enables effective test-time domain shift correction without modifying the backbone or introducing complex training objectives. Experiments on six benchmark datasets show that CO4S consistently improves generalization performance across multiple evaluation protocols, achieving an average macro-F1 gain of 5.51\% with as few as 7.83 KB of trainable parameters. These results suggest that effective text domain generalization can be achieved by better exploiting the intrinsic invariance of pretrained language models.

Paper Type: Long

Research Area: Generalizability and Transfer

Research Area Keywords: Machine Learning for NLP, representation learning, generalization

Contribution Types: Model analysis & interpretability, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 660