Improved Algorithms for Replicable Bandits

Junpei Komiyama, Shinji Ito, Yuichi Yoshida, Souta Koshino

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Interactive Learning, Reproducible Learning, Bandit Algorithms

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Improved regret of replicable bandits algorithm from $O(K^3)$ to $O(K)$

Abstract:

This work is motivated by the growing demand for reproducible machine learning. We study the stochastic multi-armed bandit problem, where the algorithm's sequence of actions is, with a high probability, not affected by the randomness of the dataset. Existing algorithms require a regret scale of $O(K^3)$, which increases much faster than the number of actions (or ``arms''), denoted as $K$. We introduce an algorithm with a distribution-dependent regret of $O(K)$. Furthermore, we propose another algorithm, which not only achieves a regret of $O(K)$ but also boasts a distribution-independent regret of $O(K^{1.5}\sqrt{T \log T})$. Additionally, we propose an algorithm for the linear bandit with regret of $O(d)$, which is linear in the dimension of associated features, denoted as $d$, and it is independent of $K$.
Our algorithms exhibit substantial simplicity compared to existing ones, and we offer a principled approach to limiting the probability of non-replication.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5545