One-stage Deep Feature Instrumental Variable Regression

Kunyang Song, Pengxin Guo, Zhoufan Zhu, Ke Zhu

Keywords: Instrumental variable, Causal inference, Supervised learning

Abstract: Instrumental variable (IV) regression is a fundamental tool for causal inference in the presence of unmeasured confounders. Traditional approaches, such as two-stage least squares and its modern extensions, rely on a two-stage learning procedure. However, this two-stage paradigm introduces inherent issues, such as error propagation, computational overhead, and sensitivity to model mis-specification. To solve these issues, we propose a novel one-stage deep feature IV (1SDFIV) regression method to directly learn the causal structural function, using a loss based on the measure of martingale difference divergence (MDD). Since MDD-type loss directly leverages the exogeneity of IV, 1SDFIV is endowed with a key characteristic of first-stage learning, bypassing the need to learn an exogenous variable from IV as required in those two-stage learning methods. Moreover, we further propose a generalized 1SDFIV (G-1SDFIV) regression method, which could achieve improved prediction performance when the causal model is mis-specified. Experimental results on both low- and high-dimensional settings show that 1SDFIV and G-1SDFIV significantly outperform their two-stage competitors, offering more accurate predictions, reduced computational burden, and greater computational stability.

Primary Area: causal reasoning

Submission Number: 11926