Batched fixed-confidence pure exploration for bandits with switching constraints

Newton Mwai, Milad Malekipirbazari, Fredrik D. Johansson

Keywords: Batched bandits, Best arm identification, switching constraints

Abstract: Many studies in multi-armed bandits focus on making exploration quick, often obliviously to constraints tied to exploration like the cost of switching between arms. Switching costs arise in many real-world settings like in healthcare when personalizing treatments, where successive assignment of the same treatment could be necessary for treatment to take effect; or in industrial applications where reconfiguring production is costly. Unfortunately, controlling for switching is significantly under-studied outside of regret minimization. In this work, we present a bandit formulation with constraints on the arm switching frequency in fixed-confidence pure exploration and give a lower bound for this setting. We present a batched bandit algorithm called SPB C-Tracking inspired by track-and-stop algorithms, adapted to batch plays with a limited number of arm switches. Finally, we demonstrate empirically that our approach achieves quick stopping times even when constrained to a minimal switching limit.

Submission Number: 85