RegCLIP: A Label-Efficient Coarse-to-Fine Learner for Ordinal Regression
Keywords: ordinal regression, contrastive learning, representation learning, vision-language
TL;DR: we propose RegCLIP, a label-efficient coarse-to-fine learner for ordinal regression.
Abstract: Ordinal regression is a fundamental problem within the field of computer vision. While pre-trained vision-language models have exhibited impressive performance on various vision tasks, their potential for ordinal regression has received less exploration.
In this paper, we introduce a novel method called RegCLIP, a label-efficient coarse-to-fine method for ordinal regression. This approach incorporates language prior information to gradually refine predictions and achieve fine-grained results.
Our RegCLIP framework encompasses two levels of coarse-to-fine concepts. The first level is a stagewise approach, performing intermediate classification initially and then refining the predictions. The second level is to generate coarse semantic labels as intermediate classes and subsequently refine them into fine-grained labels. To achieve it, we propose a novel coarse semantic label generation via large language models, which generates coarse labels. To further enhance the precision of predictions, we propose a novel fine-grained cross-modal ranking-based loss specifically designed to update fine-grained semantic labels with both semantic and ordinal alignment. Experimental results on three general ordinal regression tasks demonstrate the effectiveness of RegCLIP, exceeding state-of-the-art methods with a large margin, with 10% overall accuracy improvement on historical image dating, 1.74% overall accuracy improvement on image aesthetics assessment, and 1.33 MAE reduction on age estimation under 1-shot setting.
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 877
Loading