Test-Time Adaptation with Binary Feedback

Taeckyung Lee, Sorn Chottananurak, Junsu Kim, Jinwoo Shin, Taesik Gong, Sung-Ju Lee

TL;DR: We propose a novel test-time adaptation setting that uses few binary feedback (correct/incorrect). Our dual-path optimization algorithm, BiTTA, combines guided adaptation on uncertain samples with self-adaptation on confident predictions.

Abstract: Deep learning models perform poorly when domain shifts exist between training and test data. Test-time adaptation (TTA) is a paradigm to mitigate this issue by adapting pre-trained models using only unlabeled test samples. However, existing TTA methods can fail under severe domain shifts, while recent active TTA approaches requiring full-class labels are impractical due to high labeling costs. To address this issue, we introduce a new setting of TTA with binary feedback, which uses a few binary feedbacks from annotators to indicate whether model predictions are correct, thereby significantly reducing the labeling burden of annotators. Under the setting, we propose BiTTA, a novel dual-path optimization framework that leverages reinforcement learning to balance binary feedback-guided adaptation on uncertain samples with agreement-based self-adaptation on confident predictions. Experiments show BiTTA achieves substantial accuracy improvements over state-of-the-art baselines, demonstrating its effectiveness in handling severe distribution shifts with minimal labeling effort.

Lay Summary: AI models often make more mistakes when they see new types of data that are different from what they were trained on - this is called a distribution shift. A common way to fix this is through test-time adaptation (TTA), where the model updates itself while making predictions, using only the test data. But most TTA methods either fail in hard situations or need too much human help. This paper introduces a simple and smart solution: instead of asking people for full labels, the model only asks if its answers are right or wrong - a quick yes or no. The authors propose BiTTA, a method that learns from these yes/no answers on tricky cases and also improves itself using its own confident guesses. BiTTA works better than previous methods and even beats models that use full answers, making TTA both cheaper and more effective in the real world.

Link To Code: https://github.com/taeckyung/bitta

Primary Area: General Machine Learning->Online Learning, Active Learning and Bandits

Keywords: test-time adaptation, domain adaptation, deep learning, machine learning

Submission Number: 7203