Generative multitask learning mitigates target-causing confounding

Taro Makino, Krzysztof J. Geras, Kyunghyun Cho

Keywords: causal representation learning, causal machine learning, multitask learning, causality, robustness, out-of-distribution generalization

Abstract: We propose generative multitask learning (GMTL), a simple and scalable approach to causal machine learning in the multitask setting. Our approach makes a minor change to the conventional multitask inference objective, and improves robustness to target shift. Since GMTL only modifies the inference objective, it can be used with existing multitask learning methods without requiring additional training. The improvement in robustness comes from mitigating unobserved confounders that cause the targets, but not the input. We refer to them as \emph{target-causing confounders}. These confounders induce spurious dependencies between the input and targets. This poses a problem for conventional multitask learning, due to its assumption that the targets are conditionally independent given the input. GMTL mitigates target-causing confounding at inference time, by removing the influence of the joint target distribution, and predicting all targets jointly. This removes the spurious dependencies between the input and targets, where the degree of removal is adjustable via a single hyperparameter. This flexibility is useful for managing the trade-off between in- and out-of-distribution generalization. Our results on the Attributes of People and Taskonomy datasets reflect an improved robustness to target shift across four multitask learning methods.

TL;DR: We use ideas from causality to develop an inference objective for MTL that improves robustness to target shift.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2202.04136/code)