Falsification of Internal and External Validity in Observational Studies via Conditional Moment Restrictions

Zeshan Hussain, Ming-Chieh Shih, Michael Oberst, Ilker Demirel, David Sontag

Abstract:

Randomized Controlled Trials (RCT)s are relied
upon to assess new treatments, but suffer from
limited power to guide personalized treatment decisions. On the other hand, observational (i.e.,
non-experimental) studies have large and diverse
populations, but are prone to various biases (e.g.
residual confounding). To safely leverage the
strengths of observational studies, we focus on the
problem of falsification, whereby RCTs are used
to validate causal effect estimates learned from
observational data. In particular, we show that,
given data from both an RCT and an observational
study, assumptions on internal and external validity have an observable, testable implication in the
form of a set of Conditional Moment Restrictions
(CMRs). Further, we show that expressing these
CMRs with respect to the causal effect, or “causal
contrast”, as opposed to individual counterfactual
means, provides a more reliable falsification test.
In addition to giving guarantees on the asymptotic
properties of our test, we demonstrate superior
power and type I error of our approach on semisynthetic and real world datasets. Our approach
is interpretable, allowing a practitioner to visualize which subgroups in the population lead to
falsification of an observational study