The Sample Complexity of Best-kItems Selection from Pairwise Comparisons

Wenbo Ren, Jia Liu, Ness Shroff

Abstract: We study sample complexity bounds (i.e., bounds on the number of comparisons needed) for ranking an $n$-sized set from multi-wise comparisons. Here, a multi-wise comparison involves $m$ items and returns a (noisy) result about the best item (the winner feedback) or the order of these $m$ items (full-ranking feedback). We consider two basic ranking problems: total ranking and top-$k$ items selection. Unlike previous works that mainly focus on parametric models, in this paper, we do not require such parametric assumptions and work on the fundamental setting where each comparison returns the correct result with a certain probability larger than $1/2$. Our studies help understand whether and to what degree utilizing multi-wise comparisons can reduce the number of comparisons required for finding the ranking or partial rankings. To be specific, under the winner feedback model, one can reduce the sample complexity for top-$k$ selection up to an $m$ factor and that for full ranking up to an $\log{m}$ factor. Under the full-ranking feedback model, one can reduce the sample complexity for top-$k$ selection up to an $m$ factor and that for full ranking up to an $m\log{m}$ factor. For some cases, we prove new lower bounds to show that using multi-wise comparisons cannot reduce the sample complexity compared to that using pairwise comparisons. We also conduct simulations for the proposed algorithms and the numerical results show consistency with our theoretical results.