Adaptive Ensemble Learning With Category-Aware Attention and Local Contrastive Loss

Hongrui Guo, Tianqi Sun, Hongzhi Liu, Zhonghai Wu

Abstract: Machine learning techniques can help us deal with many difficult problems in the real world. Proper ensemble of multiple learners can improve the predictive performance. Each base learner usually has different predictive ability on different instances or in different instance regions. However, existing ensemble methods often assume that base learners have the same predictive ability for all instances without consideration of the specificity of different instances or categories. To address these issues, we propose an adaptive ensemble learning framework with category-aware attention and local contrastive loss, which can adaptively adjust the ensemble weight of each base classifier according to the characteristics of each instance. Specifically, we design a category-aware attention mechanism to learn the predictive ability of each classifier on different categories. Furthermore, we design a local contrastive loss to capture local similarities between instances and further enhance the model’s ability to discern fine-grained patterns in the data. Extensive experiments on 20 public datasets demonstrate the effectiveness of the proposed model.