Competitive Ratio and its Application in Sequential Representation Learning for Multi-task Linear Bandits

ICLR 2026 Conference Submission13737 Authors

Keywords: bandits, competitive ratio, representation learning, interactive learning, transfer learning, online learning, online pca, competitive analysis, multi-task learning, meta learning

TL;DR: We do competitive analysis on the FTL algorithm in the Online PCA setting and apply the result to Sequential representation transfer in multi-tasks linear bandit.

Abstract:

We study the competitive ratio between the cumulative loss of Follow-The-Leader (FTL) and that of the best expert in hindsight for online subset and subspace selection. In the \textit{subset selection} problem, the learner chooses a set of $s$ experts from a pool of size $K$ at each step, and we show that FTL is $K$-competitive. In the \textit{subspace selection} problem, also known as online principal component analysis, the learner chooses an $m$-dimensional subspace in $\mathbb{R}^d$ at each step, observes a context vector $x$, and incurs a ``compression loss.''
We show that FTL achieves a competitive ratio of $d$ under some mild assumptions. We apply these results to sequential representation learning in multi-task linear bandits and develop an algorithm \baron. We provide regret guarantees in the form of upper and lower bounds, and further demonstrate its computational efficiency empirically on a synthetic dataset.

Primary Area: reinforcement learning

Submission Number: 13737