Hellinger-UCB: A novel algorithm for stochastic multi-armed bandit problem

NeurIPS 2024 Workshop BDU Submission94 Authors

Keywords: Multi-armed bandit problem, Upper Confidence Bound, Squared Hellinger Distance, Cold-start problem

TL;DR: A novel algorithm for stochastic multi-armed bandit problem and application to cold start problem in recommender system.

Abstract:

In this paper, we study the stochastic multi-armed bandit problem, where the reward
is driven by an unknown random variable. We propose a new variant of the Upper
Confidence Bound (UCB) algorithm called Hellinger-UCB, which leverages the
squared Hellinger distance to build the upper confidence bound. We prove that
the Hellinger-UCB reaches the theoretical lower bound(O(T)). As a real-world
example, we apply the Hellinger-UCB algorithm to solve the cold-start problem
for a content recommender system of a financial app. With reasonable assumption,
the Hellinger-UCB algorithm has an important lower latency feature, closed-form
UCB. The online experiment also illustrates that the Hellinger-UCB outperforms
both KL-UCB and UCB1 in the sense of a higher click-through rate (CTR), 33%
higher than the KL-UCB and almost 100% higher than the UCB1.

Submission Number: 94