Efficient Wrapper Feature Selection using Autoencoder and Model Based Elimination

Sharan Ramjee, Aly El Gamal

Keywords: Wrapper Feature Selection, AMBER, Ranker Model, Generative Training, Wireless Subsampling

Abstract: We propose a computationally efficient wrapper feature selection method - called Autoencoder and Model Based Elimination of features using Relevance and Redundancy scores (AMBER) - that uses a single ranker model along with autoencoders to perform greedy backward elimination of features. The ranker model is used to prioritize the removal of features that are not critical to the classification task, while the autoencoders are used to prioritize the elimination of correlated features. We demonstrate the superior feature selection ability of AMBER on 4 well known datasets corresponding to different domain applications via comparing the accuracies with other computationally efficient state-of-the-art feature selection techniques. Interestingly, we find that the ranker model that is used for feature selection does not necessarily have to be the same as the final classifier that is trained on the selected features. Finally, we hypothesize that overfitting the ranker model on the training set facilitates the selection of more salient features.

Code: https://github.com/amber-iclr/AMBER

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1905.11592/code)