Instance-Specific Loss-Weighted Decoding for Decomposition-Based Multiclass Classification

Bin-Bin Jia, Jun-Ying Liu, Min-Ling Zhang

Abstract: Multiclass classification problems are often addressed by decomposing them into a set of binary classification tasks. A critical step in this approach is the effective aggregation of predictions from each decomposed binary classifier to yield the final multiclass prediction, a process known as decoding. Existing studies have ignored the varying generalization ability of each binary classifier across different samples during decoding, potentially leading to suboptimal performance. In this article, we propose an instance-specific loss-weighted (ILW) decoding strategy that gauges the generalization ability of each binary classifier for one specific sample based on its neighboring samples. This estimated generalization ability is then used to adjust the importance of the binary classifier in determining the sample’s final prediction. Experimental results validate the effectiveness of the ILW decoding strategy. Furthermore, we demonstrate that softmax regression can be reinterpreted as a one-versus-rest (OvR) decomposition-based multiclass classification algorithm, enabling the application of our decoding strategy to enhance its performance. Comparative studies clearly demonstrate the superiority of the improved softmax regression over its traditional counterpart.