Delayed Feedback in Generalised Linear Bandits

Benjamin Howson, Ciara Pike-Burke, Sarah Lucie Filippi

Keywords: Generalised Linear Bandits, Delayed Rewards

TL;DR: We explore the impact of delayed rewards in generalized linear bandits and propose an optimistic algorithm that achieves a regret bound where the delay penalty is independent of the decision horizon, improving upon existing work.

Abstract: The stochastic generalised linear bandit is a well-understood model for sequential decision-making problems, with many algorithms achieving near-optimal regret guarantees under immediate feedback. However, the stringent requirement for immediate rewards is unmet in many real-world applications where the reward is almost always delayed. We study the phenomenon of delayed rewards in generalised linear bandits in a theoretical manner. We show that a natural adaptation of an optimistic algorithm to the delayed feedback achieves a regret bound where the penalty for the delays is independent of the horizon. This result significantly improves upon existing work, where the best known regret bound has the delay penalty increasing with the horizon. We verify our theoretical results through experiments on simulated data.

Already Accepted Paper At Another Venue: already accepted somewhere else