Region Purity-Based Local Feature Selection: A Multiobjective Perspective

Yu Zhou, Yan Qiu, Sam Kwong

Abstract: In contrast to the traditional feature selection (FS), local FS (LFS) partitions the whole sample space and obtains the feature subset for each local region. However, most existing LFS algorithms lack a problem-specific objective function and instead simply apply the distance-like objective function, which limits their classification performance. In addition, obtaining a good LFS model is essentially a multiobjective optimization problem. Therefore, in this article, we propose a region purity (RP)-based LFS (RP-LFS) where, besides the proportion of the selected features and region-based distance metric, we design a novel objective function, RP, from the perspective of combining local features with classifiers. To solve the RP-LFS, an improved nondominated sorting genetic algorithm III is proposed. Specifically, a network-inspired crossover operator and a quick bit mutation are applied, which can improve the ability to search for better solutions. A regional feature sharing strategy between different local models is developed, which can preserve more effective features. Experimental studies on 11 UCI datasets and nine high-dimensional datasets validate the effectiveness of our proposed RP. In comparison with various state-of-the-art FS and LFS algorithms, RP-LFS can achieve very competitive classification accuracy while obtaining a reduced feature subset size.