Finding Invariant Predictors Efficiently via Causal Structure

Kenneth Lee, Md Musfiqur Rahman, Murat Kocaoglu

Keywords: Causality, invariant prediction, out-of-domain generalization

TL;DR: We develop a sound algorithm for finding invariant predictors that runs in polynomial time and yields predictive performance that is comparable to the existing work.

Abstract: One fundamental problem in machine learning is out-of-distribution generalization. A method named the surgery estimator incorporates the causal structure in the form of a directed acyclic graph (DAG) to find predictors that are invariant across target domains using distributional invariances via Pearl’s do-calculus. However, finding a surgery estimator can take exponential time as the current methods need to search through all possible predictors. In this work, we first provide a graphical characterization of the identifiability of conditional causal queries. Next, we leverage this characterization together with a greedy search step to develop a polynomial-time algorithm for finding invariant predictors using the causal graph. Given the correct causal graph, our method is guaranteed to find at least one invariant predictor, if it exists. We show that our proposed algorithm can significantly reduce the run-time both in simulated and semi-synthetic data experiments and have predictive performance that is comparable to the existing work that runs in exponential time.