Invariant Lipschitz Bandits: A Side Observation Approach

Nam Phuong Tran, Long Tran-Thanh

Abstract: Symmetry arises in many optimization and decision-making problems, and has attracted considerable attention from the optimization community: by utilizing the existence of such symmetries, the process of searching for optimal solutions can be improved significantly. Despite its success in offline settings, the utilization of symmetries has not been well examined within online optimization problems, especially in the bandit literature. As such, in this paper, we study the invariant Lipschitz bandit setting, a subclass of the Lipschitz bandits in which a group acts on the set of arms and preserves the reward function. We introduce an algorithm named UniformMesh-N, which naturally integrates side observations using group orbits into the uniform discretization algorithm [15]. Using the side-observation approach, we prove an improved regret upper bound, which depends on the cardinality of the group, given that the group is finite. We also prove a matching regret’s lower bound for the invariant Lipschitz bandit class (up to logarithmic factors). We hope that our work will ignite further investigation of symmetry in bandit theory and sequential decision-making theory in general.