Minimax Regret for Cascading Bandits

Daniel Vial, sujay sanghavi, Sanjay Shakkottai, R. Srikant

Keywords: online learning-to-rank, cascading bandits, linear stochastic bandits

Abstract: Cascading bandits is a natural and popular model that frames the task of learning to rank from Bernoulli click feedback in a bandit setting. For the case of unstructured rewards, we prove matching upper and lower bounds for the problem-independent (i.e., gap-free) regret, both of which strictly improve the best known. A key observation is that the hard instances of this problem are those with small mean rewards, i.e., the small click-through rates that are most relevant in practice. Based on this, and the fact that small mean implies small variance for Bernoullis, our key technical result shows that variance-aware confidence sets derived from the Bernstein and Chernoff bounds lead to optimal algorithms (up to log terms), whereas Hoeffding-based algorithms suffer order-wise suboptimal regret. This sharply contrasts with the standard (non-cascading) bandit setting, where the variance-aware algorithms only improve constants. In light of this and as an additional contribution, we propose a variance-aware algorithm for the structured case of linear rewards and show its regret strictly improves the state-of-the-art.

TL;DR: We prove matching (up to log terms) upper and lower bounds for the problem-independent regret of cascading bandits with unstructured rewards, and an upper bound for the case of linear rewards, that improve the best known.