Evaluating the Evaluators: Which UDA validation methods are most effective? Can they be improved?
Abstract: This paper compares and ranks 8 UDA validation methods. Validators estimate model accuracy, which makes them an essential component of any UDA train-test pipeline. We rank these validators to indicate which of them are most useful for the purpose of selecting optimal model checkpoints and hyperparameters. To the best of our knowledge, this large-scale benchmark study is the first of its kind in the UDA field. In addition, we propose 3 new validators that outperform existing validators. When paired with one particular UDA algorithm, one of our new validators achieves state-of-the-art performance.
Submission Length: Long submission (more than 12 pages of main content)
Assigned Action Editor: ~Kui_Jia1
Submission Number: 507
Loading