A Novel Ensemble Aggregation Method Based on Deep Learning Representation

Truong Thanh Nguyen, Eyad Elyan, Truong Dang, Tien Thanh Nguyen, Martin Longmuir

Abstract: We propose a novel ensemble aggregation method by using a deep learning-based representation approach. Specifically, we applied the Cross-Validation procedure on training data with a number of learning algorithms to obtain the predictions for training data called meta-data. A neural network model is trained on this meta-data to generate representations associated with class labels. In our method, the neural network model functions as an encoder, learning the relationship between base classifiers’ outputs and mapping meta-data to a representation space. The vectors in the mapped space provide a more accurate representation than traditional methods by reducing the distance of vectors in the same class and increasing the distance in different classes. Our method was compared with four well-known ensemble methods: Decision Template, an ensemble with a MultiLayer Perceptron (MLP)-based combiner, gcForest, and XgBoost. Experiments conducted on 20 UCI datasets demonstrate the outstanding performance of our ensemble aggregation method. The results show that our method achieves better delegation of class label representations, enhancing the final results of classification tasks.