Contextual Ranking and Selection with Gaussian Processes

Sait Cakmak, Enlu Zhou, Siyang Gao

Abstract: In many real world problems, we are faced with the problem of selecting the best among a finite number of alternatives, where the best alternative is determined based on context specific information. In this work, we study the contextual Ranking and Selection problem under a finite arm - finite context setting, where we aim to find the best alternative for each context. We use a separate Gaussian process to model the reward for each arm, derive the large deviations rate function for both the expected and worst-case contextual probability of correct selection, and propose an iterative algorithm for maximizing the rate function. Numerical experiments show that our algorithm is highly competitive in terms of sampling efficiency, while having significantly smaller computational overhead.