Adaptive Experimentation When You Can't Experiment

Yao Zhao, Kwang-Sung Jun, Tanner Fiez, Lalit K Jain

Keywords: Experimental Design, multi-armed bandit

Abstract:

This paper introduces the confounded pure exploration transductive linear bandit (CPET-LB) problem.
As a motivating example, often online services cannot directly assign users to specific control or treatment experiences either for business or practical reasons. In these settings, naively comparing treatment and control groups that may result from self-selection can lead to biased estimates of underlying treatment effects.
Instead, online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment.
Our methodology provides online services with an adaptive experimental design approach for learning the best-performing treatment for such encouragement designs.
We consider a more general underlying model captured by a linear structural equation and formulate pure exploration linear bandits in this setting. Though pure exploration has been extensively studied in standard adaptive experimental design settings, we believe this is the first work considering a setting where noise is confounded. Elimination-style algorithms using experimental design methods in combination with a novel finite-time confidence interval on an instrumental variable style estimator are presented with sample complexity upper bounds nearly matching a minimax lower bound. Finally, experiments are conducted that demonstrate the efficacy of our approach.

Primary Area: Bandits

Submission Number: 3159