A Unimodal, Uncertainty-Aware Deep Learning Approach for Ordinal Regression

Jonathan Svirsky, Igal Zaidman, Uri Shaham

Keywords: unimodality, ordinal regression, uncertainty, deep learning

Abstract:

Ordinal regression is an important area in machine learning, and many algorithms were proposed to approach it. In this work, we propose an ordinal regression prediction algorithm, based on deep learning machinery and inspired by the well-known Proportional Odds model. Our proposed approach has three key components: first, it is designed to guarantee unimodal output probabilities, which is a desired element in many real world applications. Second, we argue that the standard maximum likelihood is sub-optimal for ordinal regression problems and train our model using optimal transport loss, as it naturally captures the order of the classes. Third, we design a novel regularizer aiming to make the model uncertainty-aware, in the sense of making the model more confident about correct predictions, comparing to wrong predictions. In addition, we propose a novel uncertainty-awareness evaluation measure. Experimental results on eight real-world datasets
demonstrate that our proposed approach consistently performs on par with and often better than several recently proposed deep learning approaches for ordinal regression, in terms of both accuracy and uncertainty-awareness, while having a guarantee on the output unimodality.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning