Direct-Effect Risk Minimization

Yuhui Li, Zejia Wu, Chao Zhang, Hongyang Zhang

Keywords: Domain Generalization, Causal Inference, Direct and Indirect Effects

TL;DR: We develop a novel domain generalization algorithm for correlation shift based on direct causal effects, which achieves good results in our experiments on 5 correlation-shifted datasets and the DomainBed benchmark.

Abstract: We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of correlation shift and has posed concerns on the reliability of machine learning. In this work, we introduce the concepts of direct and indirect effects from causal inference to the domain generalization problem. Under mild conditions, we show that models that learn direct effects provably minimize the worst-case risk across correlation-shifted domains. To eliminate the indirect effects, our algorithm consists of two stages: in the first stage, we learn an indirect-effect representation by minimizing the prediction error of domain labels using the representation and the class label; in the second stage, we remove the indirect effects learned in the first stage by matching each data with another data of similar indirect-effect representation but of different class label. Experiments on 5 correlation-shifted datasets and the DomainBed benchmark verify the effectiveness of our approach.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Probabilistic Methods (eg, variational inference, causal inference, Gaussian processes)