Bayesian receiver operating characteristic metric for linear classifiers.

Syeda Sakira Hassan, Heikki Huttunen, Jari Niemi, Jussi Tohka

Abstract: Highlights • A new classifier accuracy metric is proposed for binary classification. • Proposed metric estimates area under the receiving operating characteristic curve. • The estimation is done directly from the training set, so no validation splits are required. • We derive a closed-form solution for this Bayesian estimator. • Experiments show that the estimator is faster and error is smaller than comparison methods. Abstract We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training dataset without the need for computationally expensive cross-validation. We derive a closed-form solution of the proposed accuracy metric for any linear binary classifier under the Gaussianity assumption, and study the accuracy of the proposed estimator using simulated and real-world data. These experiments confirm that the closed-form CBAUC is both faster and more accurate than conventional AUC estimators.