Genetic Programming-based Evolutionary Feature Construction for Heterogeneous Ensemble Learning [Hot of the Press]

Hengzhe Zhang, Aimin Zhou, Qi Chen, Bing Xue, Mengjie Zhang

Abstract: This Hof-off-the-Press paper summarizes our recently published work, "SR-Forest: A Genetic Programming based Heterogeneous Ensemble Learning Method," published in IEEE Transactions on Evolutionary Computation [4]. This paper presents SR-Forest, a novel genetic programming-based heterogeneous ensemble learning method, which combines the strengths of decision trees and genetic programming-based symbolic regression methods. Rather than treating genetic programming-based symbolic regression methods as competitors to random forests, we propose to enhance the performance of random forests by incorporating genetic programming as a complementary technique. We introduce a guided mutation operator, a multi-fidelity evaluation strategy, and an ensemble selection mechanism to accelerate the search process, reduce computational costs, and improve predictive performance. Experimental results on a regression benchmark with 120 datasets show that SR-Forest outperforms 25 existing symbolic regression and ensemble learning methods. Moreover, we demonstrate the effectiveness of SR-Forest on an XGBoost hyperparameter performance prediction task, which is an important application area of ensemble learning methods. Overall, SR-Forest provides a promising approach to solving regression problems and can serve as a valuable tool in real-world applications.