Keywords: OOD generalization, causal machine learning, geometric skew, statistical skew, classification
TL;DR: We evaluate how well geometric skews and statistical skews explain the failure of machine learning models in generalizing to unseen test domains
Abstract: Out-of-distribution (OOD) or domain generalization is the problem of generalizing to unseen distributions. Recent work suggests that the marginal difficulty of generalizing to OOD over in-distribution data (OOD-ID generalization gap) is due to spurious correlations, which arise due to statistical and geometric skews, and can be addressed by careful data augmentation and class balancing. We observe that after constructing a dataset where we remove all conceivable sources of spurious correlation between interpretable factors, classifiers still fail to close the OOD-ID generalization gap.