Contrastive Residual Energy Test-time Adaptation

ICLR 2026 Conference Submission16303 Authors

Keywords: test-time adaptation, energy-based model, calibration

TL;DR: CRETTA is a practical test-time adaptation strategy that uses a contrastive residual energy objective with adaptive gradient reweighting to achieve stable and well-calibrated robustness to distribution shifts.

Abstract: Test-Time Adaptation (TTA) enhances model robustness by enabling adaptation to target distributions that differ from training distributions, improving real-world generalizability. However, most existing TTA approaches focus on adjusting the conditional distribution and therefore exhibit poor calibration, as they rely on uncertain predictions in the absence of labels. Energy-based TTA frameworks provide an alternative by modeling the marginal distribution of target data without depending on label predictions, but their reliance on costly sampling hinders scalability in real-world scenarios where decisions must be made without latency. In this work, we propose Contrastive Residual Energy Test-time Adaptation (CRETTA), a practical solution for reliable adaptation. We first redefine the marginal distribution of target data using residual energy function and embed it into contrastive objective. This design prevents overfitting through adaptive gradient reweighting mechanism that leverages the relative residual energy while eliminating the sampling process. Extensive experiments demonstrate that CRETTA achieves scalable and well-calibrated adaptation under real-world computational constraints.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 16303