ELVES: Extraction of Latent Variables with Enhanced Specificity for High-Dimensional Few-Sample Feature Selection

Mao Li, Zhilong Mi, Yifan Cao, Qingcai He, Yin Ziqiao, GuoBinghui

Keywords: Feature Selection, Manifold Learning, Spectral Graph Theory, High-Dimensional Few-Sample Data

Abstract: Feature selection for high-dimensional, few-sample data has been a serious issue due to overfitting, high computational complexity and feature redundancy. Here, one key challenge is how to capture characterization of specificity that enhance the outcomes. To tackle this issue, our work proposes a novel supervised feature selection method named ELVES, which exploits the manifold structure of the feature space. Specifically, our method constructs a feature association kernel for each class to capture inter-feature dependencies. By integrating product manifold theory with spectral graph analysis, we develop structure operators that characterize the intrinsic geometry of each class manifold. A graph filtering operator is then designed to produce a filtered operator, whose leading eigenvectors capture class-specific latent variables. These latent variables are iteratively extracted and used to define a feature scoring mechanism that identifies features with strong discriminative power in high-dimensional, few-sample scenarios. Comprehensive experiments demonstrate that ELVES not only improves generalization performance and robustness to few sample size over leading baselines, but also provides new insights into the underlying sources of data variation.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 5721