Perceptual adjustment queries: An inverted measurement paradigm for low-rank metric learning

Austin Xu, Andrew D. McRae, Jingyan Wang, Mark A. Davenport, Ashwin Pananjady

Keywords: Metric Learning, Data Elicitation, Low-rank Matrix Estimation

TL;DR: We propose a new human query, apply it to the problem of low-rank metric learning, and prove sample complexity upper bounds in a high-dimensional setting.

Abstract: We introduce a new type of informative and yet cognitively lightweight query mechanism for collecting human feedback, called the perceptual adjustment query (PAQ). The PAQ combines advantages from both ordinal and cardinal queries. We showcase the PAQ mechanism by collecting observations on a metric space involving an unknown Mahalanobis distance, and consider the problem of learning this metric from PAQ measurements. This gives rise to a type of high dimensional, low-rank matrix estimation problem under a new measurement scheme to which standard matrix estimators cannot be applied. Consequently, we develop a two-stage estimator for metric learning from PAQs, and provide sample complexity guarantees for this estimator. We demonstrate the performance along with various properties of the estimator by extensive numerical simulations.

Submission Number: 41