Phase Transitions for One-Vs-One and One-Vs-All Linear Separability in Multiclass Gaussian Mixtures

Ganesh Ramachandra Kini, Christos Thrampoulidis

Abstract: We study a fundamental statistical question in multiclass classification: When are data linearly separable? Unlike binary classification, linear separability in multiclass settings can be defined in different ways. Here, we focus on the so called one-vs-one (OvO) and one-vs-all (OvA) linear separability. We consider data generated from a Gaussian mixture model (GMM) in a linear asymptotic high-dimensional regime. In this setting, we prove that both the OvO and OvA separability undergo a sharp phase-transition as a function of the overparameterization ratio. We present precise formulae characterizing the phase transitions as a function of the data geometry and the number of classes. Existing results on binary classification follow as special cases of our new formulae. Numerical simulations verify the validity of the asymptotic predictions in finite dimensions.