GRAPH-BASED FEATURE REPRESENTATION FOR MULTI-CLASS CLASSIFICATIONS USING THE JEFFRIES-MATUSITA DISTANCE

Anonymous

Keywords: Manifold learning, feature selection

TL;DR: A manifold learning approach for feature selection in multi-class problems, based on diffusion maps and the Jeffries-Matusita Distance

Abstract:

Visualization of the feature space has gained attention in recent years, especially with the growing
importance of explainability in AI. When processing high-dimensional datasets, a common preprocessing
step is feature selection. Filter-based feature selection algorithms are not tailored to
a specific classification method, but rather rank the relevance of each feature with respect to the
target and the task. The Jeffries-Matusita (JM) distance is a measure for separability between two
distributions that has been used for filter based feature selection. It calculates for each feature the
separability strength between pairs of classes. In this work, a low-dimensional representation of
the JM measures for the feature space is formed by diffusion map. Diffusion maps organizes the
features by their class separation abilities. Moreover, feature elimination is performed based on
the distribution of the points in the low-dimensional space. Experimental results are provided for 3
public datasets and compared with known filter-based feature selection techniques.