How to Learn a Star: Binary Classification with Starshaped Polyhedral Sets

Marie-Charlotte Brandenburg, Katharina Jochemko

Keywords: binary classification, starshaped polyhedral sets, VC dimension, convex optimization, sublevel sets, polyhedral geometry

TL;DR: We describe the geometry and combinatorics of the parameters of binary classification with starshaped polyhedral sets

Abstract: We consider binary classification restricted to a class of continuous piecewise linear functions whose decision boundaries are (possibly nonconvex) starshaped polyhedral sets, supported on a fixed polyhedral simplicial fan. We investigate the expressivity of these function classes and describe the combinatorial and geometric structure of the loss landscape, most prominently the sublevel sets, for two loss-functions: the 0/1-loss (discrete loss) and a log-likelihood loss function. In particular, we give explicit bounds on the VC dimension of this model, and concretely describe the sublevel sets of the discrete loss as chambers in a hyperplane arrangement. For the log-likelihood loss, we give sufficient conditions for the optimum to be unique, and describe the geometry of the optimum when varying the rate parameter of the underlying exponential probability distribution.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 6062