KNN+X

Daniel Gilkarov, Lee-Ad Gottlieb, Hillel Ohayon

Abstract: We introduce a new paradigm for classifying a query point based on its nearest k neighbors: Having computed the k nearest neighbors of a query, we use a learning algorithm to determine the label to be assigned to the query. This paradigm is a generalization of the well-known weighted k nearest neighbor class of algorithms, and other individual instances of it have been studied as well, for example, where the classifier used is Support Vector Machines or a neural net. Within this paradigm, we study and test new learning classifiers, and find that combining KNN with each classifier typically yields higher accuracy than using each method alone. This suggests using KNN as a pre-processing step for a wide range of familiar machine-learning algorithms.