Minimax Adaptive Online Nonparametric Regression over Besov spaces

Paul Liautaud, Pierre Gaillard, Olivier Wintenberger

Keywords: Online non-parametric regression, Adaptive learning, Wavelet-based methods, Besov spaces

Abstract:

We study online adversarial regression with convex losses against a rich class of continuous yet highly irregular competitor functions,% prediction rules,
modeled by Besov spaces $B_{pq}^s$ with general parameters $1 \leq p,q \leq \infty$ and smoothness $s > \tfrac{d}{p}$.
We introduce an adaptive
wavelet-based algorithm that performs sequential prediction without prior knowledge of $(s,p,q)$, and establish minimax-optimal regret bounds against any comparator in $B_{pq}^s$.
We further design a locally adaptive extension capable of sequentially adapting to spatially inhomogeneous smoothness. This adaptive mechanism adjusts the resolution of the predictions over both time and space, yielding refined regret bounds in terms of local regularity. Consequently, in heterogeneous environments, our adaptive guarantees can significantly surpass those obtained by standard global methods.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 15355