Online Experimentation under Privacy Induced Identity Fragmentation
Keywords: causal inference; A/B testing; treatment effect; privacy; GDPR
TL;DR: We consider global average treatment effect (GATE) estimation in A/B testing when users receive mixed treatments due to identity fragmentation.
Abstract: Randomized online experimentation is a key cornerstone for evaluating decisions for online businesses. The methodology used for estimating policy effects in online experimentation is critically dependent on user identifiers. However, nowadays consumers routinely interact with online businesses across multiple devices which are recorded with different identifiers to maintain privacy. The inability to match different device identities across consumers leads to an incorrect estimation of various causal effects. Moreover, without strong assumptions about the device-user graph, the causal effects are not identifiable. In this paper, we consider the task of estimating global treatment effects (GATE) from a fragmented view of exposures and outcomes. Experiments show that estimators obtained through our procedure are superior to standard estimators, with a lower bias and increased robustness.
Submission Number: 14
Loading