Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketching

ICLR 2026 Conference Submission14906 Authors

Keywords: Linear Bandits, Matrix Sketching, Multi-scale Sketching

TL;DR: We propose a framework for efficient sketch-based linear bandits to address the issue of linear regret that may arise with matrix sketching.

Abstract:

Linear bandits have become a cornerstone of online learning and sequential decision-making, providing solid theoretical foundations for balancing exploration and exploitation.
Within this domain, matrix sketching serves as a critical component for achieving computational efficiency, especially when confronting high-dimensional problem instances.
The sketch-based approaches reduce per-round complexity from $\Omega(d^2)$ to $O(d)$, where $d$ is the dimension.
However, this computational efficiency comes with a fundamental pitfall: when the streaming matrix exhibits heavy spectral tails, such algorithms can incur vacuous *linear regret*.
In this paper, we revisit the regret bounds and algorithmic design for sketch-based linear bandits.
Our analysis reveals that inappropriate sketch sizes can lead to substantial spectral error, severely undermining regret guarantees.
To overcome this issue, we propose Dyadic Block Sketching, a novel multi-scale matrix sketching approach that dynamically adjusts the sketch size during the learning process.
We apply this technique to linear bandits and demonstrate that the new algorithm achieves *sublinear regret* bounds without requiring prior knowledge of the streaming matrix properties.
It establishes a general framework for efficient sketch-based linear bandits, which can be integrated with any matrix sketching method that provides covariance guarantees.
Comprehensive experimental evaluation demonstrates the superior utility-efficiency trade-off achieved by our approach.

Primary Area: reinforcement learning

Submission Number: 14906