Multi-Fidelity Multi-Armed Bandits Revisited

Xuchuang Wang, Qingyun Wu, Wei Chen, John C.S. Lui

Keywords: Multi-fidelity, multi-armed bandits

TL;DR: We study the multi-fidelity multi-armed bandit problem, analyzing best arm identification and regret minimization, proposing algorithms and establishing lower and upper bounds.

Abstract:

We study the multi-fidelity multi-armed bandit ($\texttt{MF-MAB}$), an extension of the canonical multi-armed bandit (MAB) problem.
$\texttt{MF-MAB}$ allows each arm to be pulled with different costs (fidelities) and observation accuracy.
We study both the best arm identification with fixed confidence ($\texttt{BAI}$) and the regret minimization objectives.
For $\texttt{BAI}$, we present (a) a cost complexity lower bound, (b) an algorithmic framework with two alternative fidelity selection procedures,
and (c) both procedures' cost complexity upper bounds.
From both cost complexity bounds of $\texttt{MF-MAB}$,
one can recover the standard sample complexity bounds of the classic (single-fidelity) MAB.
For regret minimization of $\texttt{MF-MAB}$, we propose a new regret definition, prove its problem-independent regret lower bound $\Omega(K^{1/3}\Lambda^{2/3})$ and problem-dependent lower bound $\Omega(K\log \Lambda)$, where $K$ is the number of arms and $\Lambda$ is the decision budget in terms of cost, and devise an elimination-based algorithm whose worst-cost regret upper bound matches its corresponding lower bound up to some logarithmic terms and, whose problem-dependent bound matches its corresponding lower bound in terms of $\Lambda$.

Submission Number: 3930