Sparse Linear Bandits with Fixed Sparsity Support: Adversarial and Stochastic Regimes

Kyoungseok Jang, Nam Phuong Tran, Nicolò Cesa-Bianchi

TL;DR: We propose algorithms to solve sparse linear bandit problems in both adversarial and stochastic regimes, under the assumption of fixed sparsity support.

Abstract:

We study the problem of sparse linear bandits in both adversarial and stochastic settings. While existing literature has extensively explored sparse linear bandits in the stochastic regime, the adversarial setting, particularly for general $l_p$-ball action sets $(p>1)$, remains poorly understood. Our work addresses this gap by showing that the curse of dimensionality in adversarial linear bandits can be broken under a fixed sparsity support assumption, a natural and practical condition. Specifically, we design algorithms for the $l_\infty$- and $l_2$-balls action sets that integrate sparsity support identification with Online Stochastic Mirror Descent algorithm, achieving regret bounds of $O(s\sqrt{T}\log T )$ and $O(\sqrt{sT}\log T )$, respectively. These results nearly match the optimal results when the sparsity support is known and significantly improve upon the $ O(d\sqrt{T}) $ regret of algorithms that do not exploit the sparsity structure.
Furthermore, in the stochastic setting, we show how the geometry of the $l_p$-ball action set influences both exploration and regret, and we provide an algorithm that achieves sparsity-adaptive regret $O(s\sqrt{T}\log T )$ for large $p$. Our work highlights fundamental contrasts between adversarial and stochastic regimes and establishes the first regret guarantees for sparse adversarial linear bandits beyond the case of $l_1$-ball action set.

Submission Number: 637