Shrinkage Coefficient Estimation for Regularized Tyler's M-Estimator: A Leave One Out Approach

Karim Abou-Moustafa

Abstract:

We consider the problem of estimating a regularization parameter, or shrinkage coefficient $\alpha \in (0,1)$ for regularized Tyler M-estimators (RTME). In particular, we propose a data-dependent approach for estimating an optimal $\alpha$ based on maximizing a suitably
chosen leave-one-out cross-validated (LOOCV) likelihood function. Since the LOOCV approach scales linearly with the number of samples $n$ and hence is computationally intensive, we propose a computationally efficient approximation for the LOOCV likelihood function that permits selecting a near-optimal choice for the shrinkage coefficient $\alpha$. We demonstrate the efficiency and accuracy of our proposed approach on high-dimensional data sampled from heavy-tailed elliptical distributions, and show that it is consistently better than other methods in the literature for shrinkage coefficient estimation.