Sharp Matrix Empirical Bernstein Inequalities

Hongjian Wang, Aaditya Ramdas

Keywords: random matrix, empirical bernstein inequality, concentration inequality

TL;DR: Two closed-form empirical Bernstein inequalities that are as sharp as the non-empirical ("oracle") Bernstein inequality for matrices.

Abstract:

We present two sharp, closed-form empirical Bernstein inequalities for symmetric random matrices with bounded eigenvalues. By sharp, we mean that both inequalities adapt to the unknown variance in a tight manner: the deviation captured by the first-order $1/\sqrt{n}$ term asymptotically matches the matrix Bernstein inequality exactly, including constants, the latter requiring knowledge of the variance.
Our first inequality holds for the sample mean of independent matrices, and our second inequality holds for a mean estimator under martingale dependence at stopping times.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 6422