Multi-Source Causal Inference Using Control Variates under Outcome Selection Bias

Wenshuo Guo, Serena Lutong Wang, Peng Ding, Yixin Wang, Michael Jordan

Abstract: While many areas of machine learning have benefited from the increasing availability of large and varied datasets, the benefit to causal inference has been limited given the strong assumptions needed to ensure the identifiability of causal effects -- which are often not satisfied in real-world datasets. For example, many large observational datasets (e.g., case-control studies in epidemiology, click-through data in recommender systems) suffer from selection bias on the outcome, which makes the average treatment effect (ATE) non-identifiable. We propose an algorithm to estimate causal effects from multiple data sources, where the ATE may be identifiable only in some datasets but not others. The idea is to construct control variates across the datasets in which the ATE may not be identifiable, which provably reduces the variance of the ATE estimate. We focus on a setting where the observational datasets suffer from outcome selection bias, assuming access to an auxiliary small dataset from which we can obtain a consistent estimate of the ATE. We propose a construction of control variate by taking the difference of the conditional odds ratio estimates from the two datasets. Across simulations and two case studies with real data, we show that the control variate-based ATE estimator has consistently and significantly reduced variance against different baselines.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

For the camera-ready version, we further proofread the paper and added the acknowledgement section.

---------
For ease of reviewing, we have attached a colored “diff” pdf to the end of the main submission pdf underlining all differences between the previous version and the revised version.

Below we list the changes that were made in this revision.

**As suggested by reviewer qNNR:**
- We have added additional comments and references on the extension of the odds ratio control variate beyond binary outcomes in the revision (after Proposition 4.1 and in the proof of it). We extended the proof for Proposition 4.1 to categorial $Y$.
- We have added additional comments surrounding Theorem 4.2 to clarify that it is a general conclusion that the conditional ORs are consistent across the validation dataset and the selection-biased dataset. Theorem 4.2 just details specific coefficients for the special case of a generalized linear model.
- We have added additional comments and detailed illustrations at the end of Section 3 main text for the extension to multiple datasets.

**As suggested by reviewer j75Z:**
- We have added more detailed assumptions in Appendix A for Theorem A.1.
- We have added a note to the revised version in Section 3 on the possible different data generation processes for $\mathcal{O}_1$ and $\mathcal{O}_2$

**As suggested by reviewer ZitR:**
- We have added a note in Section 3 on assumptions for the control-variate methodology.
- We have added the reference to [1] and additional clarification to the results in [1] immediately following Proposition 4.1 in the paper.
- We created a finite sample estimator based on [3] for comparison purposes, which we detail in Appendix D.2.
- We have added a comparison to [3] in Section 5.4.1 and Figure 4.
- We have also added references to [2,3] to the introduction and related work sections.

Assigned Action Editor: ~Pierre_Alquier1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 329