Neural Networks Voting for Projection Based Ensemble Classifiers

Panagiotis Anagnostou, Petros Barmpas, Sotiris K. Tasoulis, Spiros V. Georgakopoulos, Vassilis P. Plagianakos

Abstract: Ensemble learning has been proven effective in enhancing classification accuracy by aggregating predictions from multiple base classifiers. This paper introduces a novel approach to augmenting weak projection-based classifiers using a Neural Network within a stacking ensemble framework. The proposed method capitalizes on the diverse strengths of both linear and complex models, harnessing the interpretability of projection-based classifiers, while leveraging the pattern recognition capabilities of Neural Networks. We present a comprehensive algorithm involving dataset selection, preprocessing, base model training, meta-feature generation, and Neural Network architecture design and training. Extensive experiments demonstrate the efficiency of our approach on a variety of high-dimensional biomedical datasets. Our results showcase significant accuracy improvements over standalone projection-based classifiers and conventional ensemble methods. We analyze the interpretability of the hybrid ensemble, shedding light on the insights drawn from its Neural Network component. This work not only advances the field of ensemble learning, but also underscores the potential of combining disparate classifier paradigms to achieve superior predictive performance. The code for this study is available1.1.https://github.com/panagiotisanagnostou/NNv-MRPV