Recovering Causal Features for Instrumental Variable Regression with Contrastive Learning

Gabin Agbalé, Stefan Harmeling, Alexander Marx

Keywords: Causal Representation Learning, Distribution Shift, Contrastive Learning

TL;DR: We show that contrastive learning can be leverage to perform instrument variable regression in a high-dimension and nonlinear setting.

Abstract:

Instrumental Variable (IV) regression is a standard technique for estimating causal effects in the presence of unobserved confounders. The classic IV setting assumes access to an external variable - called the instrument - which directly influences the treatment variable. In this work, we consider a more challenging yet realistic setting
where the treatment is latent, and we can only observe a nonlinear transformation of it (e.g. an image). To overcome this problem, we leverage insights from the Independently Modulated Component Analysis (IMCA), which is a framework that relaxes the independence assumption in Independent Component Analysis (ICA). Specifically, we propose a general contrastive learning framework to recover the latent treatment up to an affine transformation which may be related to the instrument by a (non-)linear function. We prove that the recovered representation is compatible with standard IV techniques. Empirically, we demonstrate the effectiveness of our method using control function and two-stage least squares (2SLS) estimators and evaluate the robustness of the learned estimators in distribution shift setting.

Submission Number: 29