An Information-Theoretic Analysis of Thompson Sampling for Logistic Bandits

Amaury Gouverneur, Borja Rodríguez Gálvez, Tobias Oechtering, Mikael Skoglund

Keywords: multi-armed bandits, logistic bandits, information-theory, Thompson Sampling, regret bounds, online optimization

Abstract:

We study the performance of the Thompson Sampling algorithm for logistic bandit problems, where the agent receives binary rewards with probabilities determined by a logistic function, $\exp(\beta \langle a, \theta \rangle)/(1+\exp(\beta \langle a, \theta \rangle))$, with slope parameter $\beta$. We focus on the setting where both the action $a$ and parameter $\theta$ lie within the $d$-dimensional unit ball. Adopting the information-theoretic framework introduced by (Russo & Van Roy, 2015), we analyze the information ratio, a statistic that quantifies the trade-off between the information gained about the optimal action and the immediate regret incurred.
We improve upon previous results by establishing that the information ratio is bounded by $\tfrac{9}{2}d\alpha^{-2}$, where $\alpha$ is a minimax measure of the alignment between the action space and the parameter space, independent of $\beta$. Notably, we derive a regret bound of order $O(d/\alpha\sqrt{T \log(\beta T/d)})$, which scales only logarithmically with the logistic function parameter $\smash{\beta}$. To the best of our knowledge, this is the first regret bound for logistic bandits that achieves logarithmic dependence on $\beta$ while being independent of the number of actions. In particular, when the action space encompasses the parameter space,
the expected regret of Thompson Sampling is of order $\tilde{O}(d \sqrt{T})$.

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3099