Spatial Sign based Direct Sparse Linear Discriminant Analysis for High Dimensional Data

Dan Zhuang, Long Feng

Keywords: High dimensional data, Linear discriminant analysis, Spatial-sign

Abstract: Robust high-dimensional classification under heavy-tailed distributions without losing efficiency, is a central challenge in modern statistics and machine learning. However, most existing linear discriminant analysis (LDA) methods are sensitive to deviations from normality and may suffer from suboptimal performance in heavy-tailed settings. This paper investigates the robust LDA problem with elliptical distributions in high-dimensional data. Our approach constructs stable discriminant directions by leveraging a robust spatial sign-based mean and covariance estimator, which allows accurate estimation even under extreme distributions. We demonstrate that SSLDA achieves an optimal convergence rate in terms of both misclassification rate and estimate error. Our theoretical results are further confirmed by extensive numerical experiments on both simulated and real datasets. Compared with state-of-the-art approaches, the SSLDA method offers superior improved finite sample performance and notable robustness against heavy-tailed distributions.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 12368