Adaptively Exploiting d-Separators with Causal Bandits

Blair Bilodeau, Linbo Wang, Daniel M. Roy

Keywords: bandit, causal bandit, adaptive, d-separation, online

TL;DR: We provide a novel algorithm that exploits causal structure when it exists while simultaneously achieving sub-linear regret in the worst case.

Abstract: Multi-armed bandit problems provide a framework to identify the optimal intervention over a sequence of repeated experiments. Without additional assumptions, minimax optimal performance (measured by cumulative regret) is well-understood. With access to additional observed variables that d-separate the intervention from the outcome (i.e., they are a d-separator), recent "causal bandit" algorithms provably incur less regret. However, in practice it is desirable to be agnostic to whether observed variables are a d-separator. Ideally, an algorithm should be adaptive; that is, perform nearly as well as an algorithm with oracle knowledge of the presence or absence of a d-separator. In this work, we formalize and study this notion of adaptivity, and provide a novel algorithm that simultaneously achieves (a) optimal regret when a d-separator is observed, improving on classical minimax algorithms, and (b) significantly smaller regret than recent causal bandit algorithms when the observed variables are not a d-separator. Crucially, our algorithm does not require any oracle knowledge of whether a d-separator is observed. We also generalize this adaptivity to other conditions, such as the front-door criterion.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2202.05100/code)