Nuisance-Robust Weighting Network for End-to-End Causal Effect Estimation

Akira Tanimoto

Primary Area: causal reasoning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: causal inference, pessimism, adversarial training

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We control for uncertainty in the nuisance estimation of propensity scores based on the pessimism principle by adversarial joint optimization of propensity model under a likelihood constraint and a stability regularizer.

Abstract:

We combine the two major approaches to causal inference: the conventional statistical approach based on weighting and the end-to-end learning with adversarial networks.
Causal inference concerns the expected loss in a distribution different from the training distribution due to intervening on the input variables.
Recently, the representation balancing approach with neural networks has repeatedly demonstrated superior performance for complex problems, owing to its end-to-end modeling by adversarial formulation.
However, some recent work has shown that the limitation lies in the unrealistic theoretical assumption of the invertibility of the representation extractor.
This inherent difficulty stems from the fact that the representation-level discrepancy in representation balancing accounts only for the uncertainty of the later layers than the representation, i.e., the hypothesis layers and the loss.
Therefore, we shed light once again on the conventional weighting-based approach, retaining the spirit of end-to-end learning.
Most conventional statistical methods are based on inverse probability weighting using propensity scores, which involves nuisance estimation of propensity as an intermediate step.
They often suffer from inaccurate estimation of the propensity scores and instability due to large weights.
One might be tempted to jointly optimize the nuisance and the target, though it may lead to an optimistic evaluation, e.g., avoiding noisy instances by weighting less when noise levels are heterogeneous.
In this paper, we propose a simple method that amalgamates the strengths of both approaches: adversarial joint optimization of the nuisance and the target.
Our formulation follows the pessimistic evaluation principle in offline reinforcement learning, which brings provable robustness to the estimation uncertainty of the nuisance and the instability due to extreme weights.
Our method performed consistently well under challenging settings with heterogeneous noise. Our code is available online: https://anonymous.4open.science/r/NuNet-002A .

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8811