The Empirical Mean is Minimax Optimal for Local Glivenko-Cantelli

Doron Cohen, Aryeh Kontorovich, Roi Weiss

TL;DR: A simple empirical mean is optimal in the Local Glivenko-Cantelli problem

Abstract: We revisit the recently introduced Local Glivenko-Cantelli setting, which studies distribution-dependent uniform convergence rates of the Empirical Mean Estimator (EME). In this work, we investigate generalizations of this setting where arbitrary estimators are allowed rather than just the EME. Can a strictly larger class of measures be learned? Can better risk decay rates be obtained? We provide exhaustive answers to these questions—which are both negative, provided the learner is barred from exploiting some infinite-dimensional pathologies. On the other hand, allowing such exploits does lead to a strictly larger class of learnable measures.

Lay Summary:

Picture an endless line of coins, each with its own (unknown) probability $p_j$ of landing heads. You flip **every** coin only $n$ times—one global round after another—and must then guess every $p_j$.

The obvious move is to use the *empirical mean*: for each coin, count the heads and divide by $n$. We prove this simple maximum-likelihood rule is *minimax-optimal* when performance is judged by the **expected maximum absolute error**—that is, the average (over the randomness of the flips) of the worst miss $\max_j |\hat p_j-p_j|$. No alternative procedure can guarantee a smaller worst-case error without extra data.

We also locate the exact “phase boundary’’: if the true biases fail a certain *local Glivenko–Cantelli* regularity, then—irrespective of ingenuity—no method can consistently estimate all $p_j$ from finitely many rounds. Thus, even when facing infinitely many parameters at once, the humble empirical mean already reaches the fundamental statistical limit.

Primary Area: Theory->Learning Theory

Keywords: Maximum Likelihood Estimator, Local Glivenko-Cantelli, Symmetry, Statistical Estimation, Minimax, Machine Learning

Submission Number: 3846