Decision Surfaces of Localized Classifiers

CScott Brown, Ryan G. Benton

Abstract: Local learning algorithms are a very general class of non-parametric lazy learners obtained by stitching together systems of locally weighted parametric models. For a system of locally-learned classifiers, there is currently no convenient method to find points on the decision surface. In this paper, we introduce a generic algorithm for finding the decision surface for systems of localized classifiers using arbitrary model families. The decision surface of a classifier is often useful for obtaining pseudo-probabilistic output from the orthogonal distance of a point to the decision surface. We therefore extend our method to find this orthogonal projection of an arbitrary point onto the decision surface for a broad class of classifier families. We specifically derive the necessary equations for computing the orthogonal projection onto the decision surface of systems of locally linear support vector machines. We demonstrate how this can be used for pseudo-probabilistic calibration, and by extension for multiclass classification strategies such as one-vs-rest or one-vs-one. Lastly, we demonstrate the efficacy of this method to obtain more accurate multi-class classifiers on popular datasets.