Meta-learning and feature ranking using genetic programming for classification: Variable terminal weighting

Anna Friedlander, Kourosh Neshatian, Mengjie Zhang

Abstract: We propose an online feature weighting method for classification by genetic programming (GP). GP's implicit feature selection was used to construct a feature weighting vector, based on the fitness of solutions in which the features were found and the frequency at which they were found. The vector was used to perform feature ranking and to perform meta-learning by biasing terminal selection in mutation. The proposed meta-learning mechanism significantly improved the quality of solutions in terms of classification accuracy on an unseen test set. The probability of success-the probability of finding the desired solution within a given number of generations (fitness evaluations)-was also higher than canonical GP. The ranking obtained by using the GP-provided feature weighting was very highly correlated with the ranking obtained by commonly used feature ranking algorithms. Population information during evolution can help shape search behaviour (meta-learning) and obtain useful information about the problem domain such as the importance of input features with respect to each other.