FraPPE: Fast and Efficient Preference-Based Pure Exploration

Udvas Das, Apurv Shukla, Debabrota Basu

Keywords: Bandits, Pareto Set Identification, Pure Exploration, Fixed Confidence, Sample Complexity, Preference Cone

TL;DR: A computationally efficient algorithm for identifying the exact Pareto optimal set with fixed confidence and any preference cone in a vector-valued Bandit. FraPPE is provably asymptotically optimal and numerically achieves the least sample complexity

Abstract: Preference-based Pure Exploration (PrePEx) aims to identify with a given confidence level the set of Pareto optimal arms in a vector-valued (aka multi-objective) bandit, where the reward vectors are ordered via a (given) preference cone $\mathcal C$. Though PrePEx and its variants are well-studied, there does not exist a *computationally efficient* algorithm that can *optimally* track the existing lower bound (Shukla and Basu, 2024) for arbitrary preference cones. We successfully fill this gap by efficiently solving the minimisation and maximisation problems in the lower bound. First, we derive three structural properties of the lower bound that yield a computationally tractable reduction of the minimisation problem. Then, we deploy a Frank-Wolfe optimiser to accelerate the maximisation problem in the lower bound. Together, these techniques solve the maxmin optimisation problem in $\mathcal O(KL^{2})$ time for a bandit instance with $K$ arms and $L$ dimensional reward, which is a significant acceleration over the literature. We further prove that our proposed PrePEx algorithm, **FraPPE**, asymptotically achieves the optimal sample complexity. Finally, we perform numerical experiments across synthetic and real datasets demonstrating that **FraPPE** achieves the lowest sample complexities to identify the exact Pareto set among the existing algorithms.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 13588