Selecting local ensembles for multi-class imbalanced data classification

Bartosz Krawczyk, Alberto Cano, Michal Wozniak

Abstract: Learning from imbalanced data is a challenge that machine learning community is facing over last decades, due to its ever-growing presence in real-life problems. While there is a significant number of works addressing the issue of handling binary and skewed datasets, its multi-class counterpart have not received as much attention. This problem is much more difficult, as presence of multiple imbalanced classes can significantly deteriorate the predictive power of any classifier. The relationship among classes are no longer clearly established and there are many difficulties embedded in the nature of such data that needs to be properly addressed. In this work, we discuss the issue of forming effective ensembles for multi-class imbalanced data based on static classifier selection approach. We propose a fully adaptive learning scheme that splits the original feature space into a number of competence areas and modifies their size and location in order to most effectively exploit the supplied pool of base classifiers. Additionally, for each established cluster we perform a weighted classifier combination, where weights are set individually for each cluster and each considered class. This allows for exploiting local competencies of each base learner in given part of feature space, as well as for each of considered classes. These two tasks are combined together in a single hybrid training scheme guided by an evolutionary algorithm. The optimization criterion is formulated in order to achieve skew-insensitive ensemble of local ensembles able to tackle highly imbalanced and multi-class problems. Experimental study proves the high efficacy of the proposed method and its superiority to other ensemble selection methods.