Dual-Phase Whitening for Test-Time Adaptation

Meng-zhu Wang

Keywords: Test-Time Adaptation, Dual-Phase Whitening, Whitening Batch Normalization, ZCA Whitening

TL;DR: DPW

Abstract: When deploying machine learning models in real-world scenarios, a key challenge is distribution shift where test data differs from the training distribution, often degarding model performance. This problem is particularly challenging in test-time adaptation (TTA), where the model must adapt to unlabeled target data without access to source data or labels. To address this problem, we introduce a novel approach to facilitate target feature learning by utilizing dual-phase whitening (DPW) in connected with whitening Batch Normalization (WBN) and whitening contrastive learning schemes (WCL). WBN operates at the feature transformation level to enforce isotropic feature distributions by ZCA whitening, thereby reducing model dependence on domain-specific covariance structures and improving stability under distribution shifts. WCL extends standard contrastive learning by incorporating global feature whitening, which eliminates redundant feature correlations while enforcing a hyperspherical distribution that better preserves semantic relationships. By the dual-phase whitening, WBN handles low-level feature standardization while WCL optimizes global representation geometry. Thus, we can obtain more generalized features from dual-phase whitening. Our method achieves state-of-the-art performance on major benchmarks including VisDA-C, DomainNet-126, ImageNet-C and CIFAR-100C have several advantages over existing works.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 4092