Leveraging Human Collaboration: Learning from Human-Verified Labels

Zhongnian Li, Shi Xu, Mengting Xu, Meng Wei, Lan Chen, Xinzheng Xu

Keywords: Human-Verified Labels, Human Collaboration

Abstract: Pre-trained Vision-Language Models (VLMs) exhibit strong zero-shot recognition accuracy and generalization capabilities, making them widely used for generating labels. However, the labels generated by VLMs are often noisy, leading to significant performance degradation for learning classifiers. To address this challenge, as shown in Figure 1, we propose a novel setting, called Human-Verified Labels (HVLs), to verify whether the labels generated by VLMs are correct with human collaboration. Specifically, HVLs enhance the quality of the labels by incorporating human verification for each label, which only needs limited labor costs. Besides, we propose a risk-consistent estimator to explore and leverage the underlying correlations between VLM-generated and human verification labels. Experimental results demonstrate the effectiveness of the proposed HVL setting.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 7008