Metric Learning with Self-Adjusting Memory for Explaining Feature Drift

Johannes Kummert, Alexander Schulz, Barbara Hammer

Abstract: Lifelong and incremental learning constitute key algorithms when dealing with streaming data in possibly non-stationary environments. Because of their capability of adapting to varying model complexity, non-parametric methods offer particularly promising methods in this realm. In this article, we focus on the self-adapting memory k-nearest neighbor classifier (SAM-kNN) as a state-of-the art method, and we elaborate on a number of recent extensions of the technology which improve its applicability in practical setups. Since the model essentially relies on a distance-based nearest neighbor classification prescription, its accuracy is limited in settings where the standard Euclidean metric is inappropriate. In this article, we introduce an extension of SAM-kNN to also incorporate metric learning, hence adjusting quadratic form according to the given data stream. Besides an improved accuracy, the model also provides better interpretability based on the induced input feature importance measures.