GL-LowPopArt: A Nearly Instance-Wise Minimax-Optimal Estimator for Generalized Low-Rank Trace Regression

Junghyun Lee, Kyoungseok Jang, Kwang-Sung Jun, Milan Vojnovic, Se-Young Yun

TL;DR: GL-LowPopArt, a new estimator for generalized low-rank trace regression, which gives improved guarantees for generalized linear matrix completion and bilinear dueling bandit, the latter being a novel setting of independent interest.

Abstract: We present `GL-LowPopArt`, a novel Catoni-style estimator for generalized low-rank trace regression. Building on `LowPopArt` (Jang et al., 2024), it employs a two-stage approach: nuclear norm regularization followed by matrix Catoni estimation. We establish state-of-the-art estimation error bounds, surpassing existing guarantees (Fan et al., 2019; Kang et al., 2022), and reveal a novel experimental design objective, $\mathrm{GL}(\pi)$. The key technical challenge is controlling bias from the nonlinear inverse link function, which we address by our two-stage approach. We prove a *local* minimax lower bound, showing that our `GL-LowPopArt` enjoys instance-wise optimality up to the condition number of the ground-truth Hessian. Applications include generalized linear matrix completion, where `GL-LowPopArt` achieves a state-of-the-art Frobenius error guarantee, and **bilinear dueling bandits**, a novel setting inspired by general preference learning (Zhang et al., 2024). Our analysis of a `GL-LowPopArt`-based explore-then-commit algorithm reveals a new, potentially interesting problem-dependent quantity, along with improved Borda regret bound than vectorization (Wu et al., 2024).

Lay Summary:

This paper asks: how many observations are needed to estimate a hidden, low-rank matrix? We answer this for generalized low-rank trace regression, applicable to diverse data like yes/no events, counts, and human preferences, where sample efficiency is crucial.

We introduce `GL-LowPopArt`, an estimator with a novel experimental design objective capturing instance-wise problem difficulty to achieve state-of-the-art theoretical error bounds. Its instance-wise guarantee adapts to each given problem instance, moving beyond pessimistic 'worst-case' minimax bounds to clarify true sample needs. We also prove that our algorithm attains near instance-wise minimax optimality.

Key applications: 1) Improved sample complexity for generalized linear matrix completion in recommender systems. 2) Introducing and solving **bilinear dueling bandits**, a novel problem from contextual preference learning that models potentially intransitive human preferences with item features. Our approach gives strong statistical guarantees for this setting.

Though theoretically tractable, `GL-LowPopArt`'s reliance on SVD of vectorized covariance creates memory/computation challenges for large matrices. Overcoming this is future work. Its instance-wise guarantees should spur more practical, improved algorithms.

Link To Code: https://github.com/nick-jhlee/GL-LowPopArt

Primary Area: Theory->Learning Theory

Keywords: generalized linear model, trace regression, Catoni's estimator, experimental design, local minimax lower bound, matrix completion, low-rank bandits, dueling bandits

Submission Number: 12122