Borda Regret Minimization for Generalized Linear Dueling Bandits
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Borda score, dueling bandit, generalized linear model, linear bandit, regret minimization
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We consider a new formulation of generalized linear dueling bandits. We show matching upper and lower bounds for the Borda regret.
Abstract: Dueling bandits are widely used to model preferential feedback prevalent in many applications such as recommendation systems and ranking.
In this paper, we study the Borda regret minimization problem for dueling bandits, which aims to identify the item with the highest Borda score while minimizing the cumulative regret.
We propose a rich class of generalized linear dueling bandit models, which cover many existing models.
We first prove a regret lower bound of order $\Omega(d^{2/3} T^{2/3})$ for the Borda regret minimization problem, where $d$ is the dimension of contextual vectors and $T$ is the time horizon.
To attain this lower bound, we propose an explore-then-commit type algorithm for the stochastic setting, which has a nearly matching regret upper bound $\tilde{O}(d^{2/3} T^{2/3})$.
We also propose an EXP3-type algorithm for the adversarial setting, where the underlying model parameter can change at each round. Our algorithm achieves an $\tilde{O}(d^{2/3} T^{2/3})$ regret, which is also optimal.
Empirical evaluations on both synthetic data and a simulated real-world environment are conducted to corroborate our theoretical analysis.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
Supplementary Material: zip
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4042
Loading