MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models

Yuqing Lei, Yingjun Du, Yawen Huang, Xiantong Zhen, Ling Shao

Track: Regular papers (within 8 pages excluding appendix)

Keywords: vision-language models, test-time adaptation, meta-learning

Abstract: Vision-language models (VLMs) such as CLIP exhibit strong zero-shot generalization but remain sensitive to domain shifts at test time. Test-time prompt tuning (TPT) mitigates this issue by adapting prompts with fixed augmentations, which may falter in more challenging settings. In this work, we propose Meta Test-Time Prompt Tuning (MetaTPT), a meta-learning framework that learns a self-supervised auxiliary task to guide test-time prompt tuning. The auxiliary task dynamically learns parameterized augmentations for each sample, enabling more expressive transformations that capture essential features in target domains. MetaTPT adopts a dual-loop optimization paradigm: an inner loop learns a self-supervised task that generates informative views, while the outer loop performs prompt tuning by enforcing consistency across these views. By coupling augmentation learning with prompt tuning, MetaTPT improves test-time adaptation under domain shifts. Extensive experiments demonstrate that MetaTPT achieves state-of-the-art performance on domain generalization and cross-dataset benchmarks.

Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.

Submission Number: 2