Hierarchical Ensemble Based Imbalance Classification

Jie Xie, Mingying Zhu, Kai Hu

Abstract: In this paper, we propose a hierarchical ensemble method for improved imbalance classification. Specifically, we perform the first-level ensemble based on bootstrap sampling with replacement to create an ensemble. Then, the second-level ensemble is generated based on two different weighting strategies, where the strategy having better performance is selected for the subsequent analysis. Next, the third-level ensemble is obtained via the combination of two methods for obtaining mean and covariance of multivariate Gaussian distribution, where the oversampling is then realized via the fitted multivariate Gaussian distribution. Here, different subsets are created by (1) the cluster that the current instance belongs to, and (2) the current instance and its k nearest minority neighbors. Furthermore, Euclidean distance-based sample optimization is developed for improved imbalance classification. Finally, late fusion based on majority voting is utilized to obtain final predictions. Experiment results on 15 KEEL datasets demonstrate the great effectiveness of our proposed method.