Label-encoding Risk Minimization under Label Insufficient Scenarios

Yulong Zhang, Yuan Yao, Shuhao Chen, Pengrong Jin, Yu Zhang, Jian Jin, Jiangang Lu

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Label insufficient scenario, semi-supervised learning, domain adaptation, heterogeneous domain adaptation, label encoding

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose label-encoding risk minimization to deal with label insufficient scenarios, including semi-supervised learning and domain adaptation.

Abstract: The Empirical Risk Minimization (ERM) adopts the supervision information, $i.e.$, class labels, to guide the learning of labeled samples and achieves great success in many applications. However, many real-world applications usually face the label insufficient scenario, where there exist limited or even no labeled samples but abundant unlabeled samples. Under those scenarios, the ERM cannot be directly applied to tackle them. To alleviate this issue, we propose a Label-encoding Risk Minimization (LRM), which draws inspiration from the phenomenon of neural collapse. Specifically, the proposed LRM first estimates the label encodings through prediction means for unlabeled samples and then aligns them with their corresponding ground-truth label encodings. As a result, the LRM takes both the prediction discriminability and diversity into account and can be utilized as a plugin in existing models to address scenarios with insufficient labels. Theoretically, we analyze the relationship between the LRM and ERM. Empirically, we demonstrate the superiority of the LRM under several label insufficient scenarios, including semi-supervised learning, unsupervised domain adaptation, and semi-supervised heterogeneous domain adaptation. The code will be released soon.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5287