Invariance-Based Causal Estimation in the Presence of Concept Drift

Katie Matton, John Guttag, Rosalind Picard

Keywords: causal estimation, invariance, IRM, concept drift

TL;DR: We adapt IRM to enable invariance-based causal estimation in settings that exhibit concept drift.

Abstract: Machine learning models are prone to relying on spurious correlations. Recently, there has been substantial progress towards solving this problem using invariant learning methods. These methods exploit the invariance of causal mechanisms across environments to distinguish between causal and spurious parts of the feature space. Existing methods have produced impressive results in constrained settings, but rely on assumptions that limit their applicability to real-world problems. In this work, we relax one of these assumptions: the absence of concept drift. We examine a simple case of concept drift, in which the label distribution is influenced by environment-dependent constant-mean shifts. We show that in this setting, existing methods fail. We then present a new method, called alternating invariant risk minimization (AIRM), that solves the problem. It works by alternating between using invariant risk minimization to learn a causal representation, and using empirical risk minimization to learn environment-specific shift parameters. We evaluate AIRM on two synthetic datasets, and show that it outperforms baselines.