Robust gradient aware and reliable entropy minimization for stable test-time adaptation in dynamic scenarios

Haoyu Xiong, Yu Xiang

Abstract:

Test-time adaptation (TTA) aims to provide neural networks capable of adapting to the target domain distribution using
only unlabeled test data. Most existing TTA methods have achieved success under mild conditions, such as independently
sampled data from a single or multiple static domains. However, these attempts may fail in dynamic scenarios, where the
test data distribution undergoes continuous changes over time. By digging into the failure cases, we find that high-entropy or
noisy samples during long-term adaptation may lead to inevitable catastrophic failure. Thus, we propose a Robust Gradient
Aware and Reliable entropy minimization approach, called RGAR, to further stabilize TTA from three aspects: (1) Boosting
model robustness to distribution shift, we propose a dual-stream perturbation technique that enables two weak-to-strong
perturbation views of the student model guided by a common strong view of the mean teacher model; (2) mitigating the
impact of high-entropy samples from different scenarios, we present to minimize the reliable samples that take into account
both the distribution shift and sample adaptation degree; (3) enabling the model to be insensitive to small perturbations by
encouraging model weights to reach flatter minima while focusing on the maximal gradient norm. Extensive experimental
results demonstrate the effectiveness of our proposed method, RGAR. We achieve state-of-the-art performance on widely used
benchmark datasets, such as CIFAR10C, CIFAR100C, and ImageNet-C. Our source code is available at https://anonymous.
4open.science/r/D152/.