Bags of Projected Nearest Neighbours: Competitors to Random Forests?

David P. Hofmeyr

Abstract: In this paper we introduce a simple and intuitive adaptive k nearest neighbours classifier, and explore its utility within the context of bootstrap aggregating (“bagging”). The approach is based on finding discriminant subspaces which are computationally efficient to compute, and are motivated by enhancing the discrimination of classes through nearest neighbour classifiers. This adaptiveness promotes diversity of the individual classifiers fit across different bootstrap samples, and so further leverages the variance reducing effect of bagging. Extensive experimental results are presented documenting the strong performance of the proposed approach in comparison with Random Forest classifiers, as well as other nearest neighbours based ensembles from the literature, plus other relevant benchmarks.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: No further changes

Code: https://github.com/DavidHofmeyr/BOPNN

Assigned Action Editor: ~Andres_R_Masegosa1

Submission Number: 5473