Robust Estimation Under Heterogeneous Corruption Rates

Syomantak Chaudhuri, Jerry Li, Thomas Courtade

Keywords: robust statistics, robust estimation, robust regression, minimax estimation

TL;DR: Minimax rate of robust estimation when different samples have different but known rates of corruption.

Abstract:

We study the problem of robust estimation under heterogeneous corruption rates, where each sample may be independently corrupted with a known but non-identical probability.
This setting arises naturally in distributed and federated learning, crowdsourcing, and sensor networks, yet existing robust estimators typically assume uniform or worst-case corruption, ignoring structural heterogeneity.
For mean estimation for multivariate bounded distributions and univariate gaussian distributions, we give tight minimax rates for all heterogeneous corruption patterns.
For multivariate gaussian mean estimation and linear regression, we establish the minimax rate for squared error up to a factor of $\sqrt{d}$, where $d$ is the dimension.
Roughly, our findings suggest that samples beyond a certain corruption threshold may be discarded by the optimal estimators -- this threshold is determined by the empirical distribution of the corruption rates given.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 10779