ERM++: An Improved Baseline for Domain Generalization
Keywords: Domain Generalization
TL;DR: An improved baseline for Domain Generalization
Abstract: Multi-source Domain Generalization (DG) measures a classifier's ability to generalize to new distributions of data it was not trained on, given several training domains. While several multi-source DG methods have been proposed, they incur additional complexity during training by using domain labels. Recent work has shown that a well-tuned Empirical Risk Minimization (ERM) training procedure, that is simply minimizing the empirical risk on the source domains, can outperform most existing DG methods. We identify several key candidate techniques to further improve ERM performance, such as better utilization of training data, model parameter selection, and weight-space regularization. We call the resulting method ERM++, and show it significantly improves the performance of DG on five multi-source datasets by over 5% compared to standard ERM, and beats state-of-the-art despite being less computationally expensive. We hope that ERM++ becomes a default baseline for DG. Code is released at https://github.com/piotr-teterwak/erm_plusplus.
Submission Number: 64
Loading