Evaluating the Impact of Geometric and Statistical Skews on Out-Of-Distribution Generalization Performance

Aengus Lynch, Jean Kaddour, Ricardo Silva

Keywords: OOD generalization, causal machine learning, geometric skew, statistical skew, classification

TL;DR: We evaluate how well geometric skews and statistical skews explain the failure of machine learning models in generalizing to unseen test domains

Abstract: Out-of-distribution (OOD) or domain generalization is the problem of generalizing to unseen distributions. Recent work suggests that the marginal difficulty of generalizing to OOD over in-distribution data (OOD-ID generalization gap) is due to spurious correlations, which arise due to statistical and geometric skews, and can be addressed by careful data augmentation and class balancing. We observe that after constructing a dataset where we remove all conceivable sources of spurious correlation between interpretable factors, classifiers still fail to close the OOD-ID generalization gap.