UNICORNN: Unimodal Calibrated Ordinal Regression Neural Network

TMLR Paper4776 Authors

Abstract:

Ordinal regression is a supervised machine learning technique aimed at predicting the value of a discrete dependent variable with an ordered set of possible outcomes. Many of the algorithms that have been developed to address this issue rely on maximum likelihood for training. However, the standard maximum likelihood approach often fails to adequately capture the inherent order of classes, even though it tends to produce well-calibrated probabilities. Alternatively, some methods use Optimal Transport (OT) divergence as their training objective. Unlike maximum likelihood, OT accounts for the ordering of classes; however, in this manuscript, we show that it doesn't always yield well-calibrated probabilities. To overcome these limitations, we introduce UNICORNN, an approach inspired by the well-known Proportional Odds Model, which offers three key guarantees: (i) it ensures unimodal output probabilities, a valuable feature for many real-world applications;
(ii) it employs OT loss during training to accurately capture the natural order of classes;
(iii) it provides well-calibrated probability estimates through a post-training accuracy-preserving calibration step.
Experimental results on six real-world datasets
demonstrate that UNICORNN consistently either outperforms or performs as well as recently proposed deep learning approaches for ordinal regression. It excels in both accuracy and probability calibration, while also guaranteeing output unimodality. The code will be publicly available upon acceptance.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Hsuan-Tien_Lin1

Submission Number: 4776