Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice
Keywords: preference learning, human-in-the-loop, AI-assistance for decision making, user modeling, cogntitive science, retrosynthesis planning
TL;DR: We improve learning of latent utilities from preferences for decision tasks, by using a cognitive model of preferential choice which models various context effects.
Abstract: Preference learning methods make use of models of human choice in order to infer the latent utilities that underlie human behaviour. However, accurate modeling of human choice behavior is challenging due to a range of context effects that arise from how humans contrast and evaluate options. Cognitive science has proposed several models that capture these intricacies but, due to their intractable nature, work on preference learning has, in practice, had to rely on tractable but simplified variants of the well-known Bradley-Terry model. In this paper, we take one state-of-the-art intractable cognitive model and propose a tractable surrogate that is suitable for deployment in preference learning. We then introduce a mechanism for fitting the surrogate to human data that cannot be explained by the original cognitive model. We demonstrate on large-scale human data that this model produces significantly better inferences on static and actively elicited data than existing Bradley-Terry variants. We further show in simulation that when using this model for preference learning, we can significantly improve a utility in a range of real-world tasks.
Primary Area: Human-AI interaction
Submission Number: 18816
Loading