Performance Metric Elicitation from Pairwise Classifier Comparisons

Gaurush Hiranandani, Shant Boodaghians, Ruta Mehta, Oluwasanmi Koyejo

Abstract: Given a binary prediction problem, which performance metric should the classifier optimize? We address this question by formalizing the problem of Metric Elicitation. The goal of metric elicitation...