Metric Learning-Guided Least Squares Classifier Learning

Chuanxing Geng, Songcan Chen

Abstract: For a multicategory classification problem, discriminative least squares regression (DLSR) explicitly introduces an ε-dragging technique to enlarge the margin between the categories, yielding superior classification performance from a margin perspective. In this brief, we reconsider this classification problem from a metric learning perspective and propose a framework of metric learning-guided least squares classifier (MLG-LSC) learning. The core idea is to learn a unified metric matrix for the error of LSR, such that such a metric matrix can yield small distances for the same category, while large ones for the different categories. As opposed to the ε-dragging in DLSR, we call this the errordragging (e-dragging). Different from DLSR and its related variants, our MLG-LSC implicitly carries out the e-dragging and can naturally reflect the roughly relative distance relationships among the categories from a metric learning perspective. Furthermore, our optimization objective functions are strictly (geodesically) convex and thus can obtain their corresponding closed-form solutions, resulting in higher computational performance. Experimental results on a set of benchmark data sets indicate the validity of our learning framework.