Test Time Adaptation Using Adaptive Quantile Recalibration

Paria Mehrbod, Pedro Vianna, geraldin nanfack, Guy Wolf, Eugene Belilovsky

Keywords: test-time adaptation, domain adaptation, domain shift, test-time distribution shift

Abstract:

Domain adaptation methods have emerged as effective mechanisms to improve the generalizability and robustness of deep learning models, particularly in real-world scenarios where test data may differ significantly from the training domain. However, traditional domain adaptation techniques often require prior knowledge of target domains or model retraining, which limits their applicability in dynamic settings where such information is unavailable or retraining is impractical.
Approaches based on updating batch normalization statistics at test-time have been gaining traction, as it allows for unsupervised adaptation based on the target data. Some of these approaches only adjust batch normalization statistics and do not fully capture complex distributions and are restricted to specific normalization types. To address this, we propose Adaptive Quantile Recalibration (AQR), a novel test-time adaptation method based on quantile recalibration, which modifies the pre-activation distributions by aligning quantiles on a channel-by-channel basis. AQR captures the complete shape of activation distributions and works across diverse architectures regardless of normalization type (BatchNorm, GroupNorm, or LayerNorm).We demonstrate that our method provides robust adaptation across diverse settings, outperforming baseline test-time adaptation methods.

Submission Number: 58