Identification of the Generalized Condorcet Winner in Multi-dueling Bandits

Björn Haddenhorst, Viktor Bengs, Eyke Hüllermeier

Keywords: Best arm identification, Condorcet Winner, Dvoretzky–Kiefer–Wolfowitz inequality, Multi-armed Bandits, Preference Learning

TL;DR: We prove (up to logarithmic terms) optimal upper and lower bounds on the sample complexity for finding with high confidence the generalized Condorcet winner in the preference-based multi-armed bandits setting

Abstract: The reliable identification of the “best” arm while keeping the sample complexity as low as possible is a common task in the field of multi-armed bandits. In the multi-dueling variant of multi-armed bandits, where feedback is provided in the form of a winning arm among as set of k chosen ones, a reasonable notion of best arm is the generalized Condorcet winner (GCW). The latter is an the arm that has the greatest probability of being the winner in each subset containing it. In this paper, we derive lower bounds on the sample complexity for the task of identifying the GCW under various assumptions. As a by-product, our lower bound results provide new insights for the special case of dueling bandits (k = 2). We propose the Dvoretzky–Kiefer–Wolfowitz tournament (DKWT) algorithm, which we prove to be nearly optimal. In a numerical study, we show that DKWT empirically outperforms current state-of-the-art algorithms, even in the special case of dueling bandits or under a Plackett-Luce assumption on the feedback mechanism.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/bjoernhad/GCWidentification