A Decomposition based Multi-objective Evolutionary Algorithm with ReliefF based Local Search and Solution Repair Mechanism for Feature Selection

Kaan Demir, Bach Hoai Nguyen, Bing Xue, Mengjie Zhang

Abstract: Feature selection has two main objectives which are to maximise the classification accuracy and to minimise the number of selected features. Unfortunately, the two objectives are usually in conflict, which makes feature selection a multi-objective problem. MOEA/D (multi-objective optimisation evolutionary algorithm based on decomposition) has shown to be effective in solving multi-objective feature selection, which evolves more diverse fronts than other multi-objective algorithms such as SPEA2 or NSGAII. However, sometimes the feature subsets around the middle of the evolved fronts do not have high classification performance. The goal of this work is to propose a local search for MOEA/D with an expectation of maintaining the front diversity while improving the classification performance of the feature subsets in the evolved fronts. The local search is based on three operators: insert, remove, and swap. The insert/remove operators either add/remove a single feature from the current feature subset, while the swap operator exchanges a selected feature with an unselected feature. The selection of added/removed/swapped features is based on Relief, a well-known measure which considers feature interactions. The experimental results show that the proposed local search can maintain or improve the fronts evolved by MOEA/D-DYN, a state-of-the-art MOEA/D algorithm for feature selection.