Rank4Class: Examining Multiclass Classification through the Lens of Learning to Rank

Nan Wang, Zhen Qin, Le Yan, Honglei Zhuang, Xuanhui Wang, Michael Bendersky, Marc Najork

Keywords: multiclass classification, learning to rank, neural ranking models

Abstract: Multiclass classification (MCC) is a classical machine learning problem which aims to classify each instance into one of a predefined set of classes. Given an instance, a classification model computes a score for each class, all of which are then used to sort the classes. The performance of a classification model is usually measured by Top-K Accuracy (e.g., K=1 or 5). In this paper, we examine MCC through the lens of learning to rank (LTR) in the deep learning setting. By viewing MCC as to rank classes for an instance, we first argue that ranking metrics from the information retrieval literature, such as Normalized Discounted Cumulative Gain (NDCG), can be more informative than the existing Top-K metrics in evaluating the performance of classification models, especially for real-world user-facing applications. We further demonstrate that the most popular MCC architecture in deep learning can be mathematically formulated as a LTR pipeline equivalently, with a specific set of choices in terms of ranking model architecture and loss function. Based on these observations, we propose several techniques, stemmed from the rich LTR literature, to improve the MCC performance. Comprehensive empirical results on both text and image classification tasks, with diverse datasets and backbone models (e.g., BERT for text classification and ResNet for image classification) show the value of our proposed framework.

One-sentence Summary: In this paper, we examine the multiclass classification problem from the lens of learning to rank to inspire more flexible design options for improved performance.