Dynamic Elimination For PAC Optimal Item Selection From Relative Feedback

En Zhi Tan

Keywords: probably approximately correct, optimal item selection, relative feedback, multi armed bandits, Plackett Luce Model, Condorcet winner, Bayesian updates, active learning

TL;DR: Algorithms for PAC optimal item selection from subset wise relative feedback based on suboptimal item dynamic elimination that outperforms SOTA benchmarks; additionally introduce the notion of inferred updates to utilize item similarity information

Abstract:

We study the problem of best-item identification from relative feedback where a learner adaptively plays subsets of items and receives stochastic feedback in the form of the best item in the set. We propose an algorithm - Dynamic Elimination (DE) - that dynamically prunes sub-optimal items from contention to efficiently identify the best item and show a strong sample complexity upper bound for it. We further formalize the notion of inferred updates to obtain estimates on item win rates without directly playing them by leveraging item correlation information. We propose the Dynamic Elimination by Correlation (DEBC) algorithm as an extension to DE with inferred updates. We show through extensive experiments that DE and DEBC significantly outperform all existing baselines across multiple datasets in various settings.

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9267