Monitoring Risks in Test-Time Adaptation

Mona Schirmer, Metod Jazbec, Christian A. Naesseth, Eric Nalisnick

Keywords: Risk Control, Test-time Adaptation

TL;DR: We introduced an unsupervised risk monitoring tool for TTA based on sequential testing.

Abstract: Encountering distribution shift at test time is a common challenge for deployed models. Test-time adaptation (TTA) addresses this by adapting models online using only unlabeled test data. While TTA can prolong a model’s usefulness, it may eventually fail, requiring the model to be taken offline and retrained. We propose augmenting TTA with a risk monitoring framework that raises alarms when performance degrades beyond a predefined threshold. Our method extends sequential testing with confidence sequences to support model updates and operate without test labels. We validate our approach across diverse datasets, shift types, and TTA methods.

Submission Number: 72