Inferring Feature Relevances From Metric Learning

Alexander Schulz, Bassam Mokbel, Michael Biehl, Barbara Hammer

Abstract: Powerful metric learning algorithms have been proposed in the last years which do not only greatly enhance the accuracy of distance-based classifiers and nearest neighbor database retrieval, but which also enable the interpretability of these operations by assigning explicit relevance weights to the single data components. Starting with the work [1], it has been noticed, however, that this procedure has very limited validity in the important case of high data dimensionality or high feature correlations: the resulting relevance profiles are random to a large extend, leading to invalid interpretation and fluctuations of its accuracy for novel data. While the work [1] proposes a first cure by means of L2-regularisation, it only preserves strongly relevant features, leaving weakly relevant and not necessarily unique features undetected. In this contribution, we enhance the technique by an efficient linear programming scheme which enables the unique identification of a relevance interval for every observed feature, this way identifying both, strongly and weakly relevant features for a given metric.