Linear diffusion models meet contextual bandits with large action spaces
Keywords: Contextual Bandits, Linear Diffusion Models
TL;DR: We use linear diffusion models to make contextual Thompson sampling more efficient and scalable in large action spaces. Both theory and algorithmic foundations are developed.
Abstract: Efficient exploration is a key challenge in contextual bandits due to the potentially large size of their action space, where uninformed exploration can result in computational and statistical inefficiencies. Fortunately, the rewards of actions are often correlated and this can be leveraged to explore them efficiently. In this work, we capture such correlations using pre-trained linear diffusion models; upon which we design diffusion Thompson sampling (dTS). Both theoretical and algorithmic foundations are developed for dTS, and empirical evaluation also shows its favorable performance.
Submission Number: 16
Loading