Quantile Multi-Armed Bandits with 1-bit Feedback
Abstract: In this paper, we study a variant of best-arm identification consisting of elements of risk sensitivity and communication constraints. Specifically, the goal of the learner is to identify the arm with the highest quantile reward, while the communication between the agent and the learner is restricted to only one bit of feedback per arm pull. We propose an algorithm that utilizes noisy binary search as a subroutine, allowing the learner to estimate quantile rewards through 1-bit feedback. We derive an instance-dependent upper bound on the sample complexity of our algorithm and provide an algorithm-independent lower bound for specific instances, with two matching to within logarithmic factors under mild conditions, or even to within constant factors in certain low error probability scaling regimes. Thus, we conclude that restricting to 1-bit feedback has a minimal impact on the scaling of the sample complexity.
Submission Number: 33
Loading