Nonparametric Different-Feature Selection Using Wasserstein Distance

Wenbo Zheng, Fei-Yue Wang, Chao Gou

Abstract: In this paper, we propose a feature selection method that characterizes the difference between two kinds of probability distributions. The key idea is to view the feature selection problem as a sparsest k-subgraph problem that considers Wasserstein distance between the studied two probability distributions. Our method does not presume any specific parametric models on the data distribution and is non-parametric. It outperforms existing Kullback-Leibler divergence based approaches, since we do not require two distributions to overlap. This relaxation makes our method work in many problems in which Kullback-Leibler divergence based methods fail. We also design a fast calculation algorithm using dynamic programming. Our experimental results show that our method outperforms the current method in both computation accuracy and speed.