Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks

Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond Veldhuis, Decebal Constantin Mocanu

Abstract: Feature selection that selects an informative subset of variables from data not only enhances the model interpretability and performance but also alleviates the resource demands. Recently, there has been growing attention in feature selection using neural networks. However, existing methods usually suffer from high computational costs when applied to high-dimensional datasets. In this paper, inspired by evolution processes, we propose a novel resource-efficient supervised feature selection method using sparse neural networks, named "NeuroFS". By gradually pruning the uninformative features from the input layer of a sparse neural network trained from scratch, NeuroFS derives an informative subset of features efficiently. By performing several experiments on $11$ low and high-dimensional real-world benchmarks of different types, we demonstrate that NeuroFS achieves the highest ranking-based score among the considered state-of-the-art supervised feature selection models. We will make the code publicly available on GitHub after acceptance of the paper.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: The changes have been colored in blue in the manuscript.

Code: https://github.com/zahraatashgahi/NeuroFS

Assigned Action Editor: ~Makoto_Yamada3

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 463