Confidence Calibration in Source-Free Domain Adaptation based on Pseudo-Labels

Rotem Nizhar, Coby Penso, Idit Diamant, Jacob Goldberger

Keywords: confidence calibration, domain adaptation, pseudo-labels

TL;DR: We suggest a confident calibration method for the source-free domain adaptation setup which is based on pseudo-labels.

Abstract:

In this study, we explore the setting of source-free domain adaptation where access to labeled data from the source domain is restricted.
We address the challenges associated with calibrating the prediction uncertainty of the adapted network solely based on unlabeled data from the target domain. To approximate the unknown true labels, we use pseudo-labels generated by the source model. Despite the high noise level in pseudo-labels, our empirical analysis reveals that the network’s accuracy computed using them closely matches the accuracy obtained with the true labels. Based on this observation, we propose a strategy for source-free confidence calibration. Our method is evaluated on standard domain adaptation benchmarks and achieves performance comparable to, or even better than, methods that require access to source data. Moreover, we significantly improve upon the state-of-the-art in source-free confidence calibration methods.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 7991