Test-time Zero-shot Recognition with Good Attributes
Keywords: Test-time adaptation, prompt learning, attribute search, soft voting, vision recognition
Abstract: Test-time adaptation (TTA) has emerged as a zero-shot learning approach to address distribution shifts across domains without needing source data. While current methods focus on adapting vision and language models (VLMs) using prompt tuning, they struggle with ambiguous categories due to the challenge of selecting relevant attributes in the absence of labels. To address this issue, we propose a novel framework, termed Search4Prompt, which aims to identify "good'' attributes and learn tailored prompts during test-time prompt learning (TTPL). Search4Prompt consists of two main components: the Retrieve-based Attribute Search (RAS) and the Implicit-Explicit Attribute Injection (IEAI) module. RAS constructs an attribute bank by generating detailed descriptions for predefined categories, and then identifies the most relevant attributes based on the semantic similarity between the test image and the attributes. This enables the selection of "good" attributes that are well-suited to the test samples. The IEAI module operates in two ways. First, it employs pseudo-label learning, where the selected attributes contribute to a voting process that implicitly injects attribute knowledge into prompt learning. Second, it augments the original category names with the selected attributes, explicitly enhancing the semantic representation of ambiguous categories. This dual approach improves the model's discriminability during test-time prompt learning. Experimental results demonstrate that Search4Prompt outperforms existing TTA methods on several benchmark datasets, confirming its effectiveness in narrowing domain gaps and handling ambiguous categories.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4218
Loading