On robust regression with high-dimensional predictors

Noureddine El Karoui, Derek Bean, Peter Bickel, Bin Yu

Abstract: We study regression M-estimates in the setting where p, the number of covariates, and n, the number of observations, are both large, but. We find an exact stochastic representation for the distribution of at fixed p and n under various assumptions on the objective function ρ and our statistical model. A scalar random variable whose deterministic limit can be studied when plays a central role in this representation. We discover a nonlinear system of two deterministic equations that characterizes. Interestingly, the system shows that depends on ρ through proximal mappings of ρ as well as various aspects of the statistical model underlying our study. Several surprising results emerge. In particular, we show that, when is large enough, least squares becomes preferable to least absolute deviations for double-exponential errors.