Beyond Similarity for Personalization: User Memory Selection via Response-Utility Optimization

ICLR 2026 Conference Submission22680 Authors

Keywords: Personalization Method, Efficient Methods, Information-Theoretics

TL;DR: We introduce a response-utility optimized method for memory selection for personalization.

Abstract: A common approach to personalization in large language models (LLMs) is to incorporate a subset of the user memory into the prompt at inference time to guide the model's generation. Existing methods to select these subsets primarily rely on similarity between user memory items and input queries, ignoring how these items actually affect the model's predictive distribution. We propose **R**esponse-**U**tility optimization for **M**emory **S**election (RUMS), a novel user memory selection method, inspired by Bayesian Optimal Experimental Design, that directly quantifies how much each memory item reduces uncertainty in the model's response distribution. RUMS measures mutual information between a subset of user memory and model outputs to identify items that sharpen predictions beyond semantic similarity. Even more, RUMS, by design, automatically selects if personalization is beneficial at all. We demonstrate that this information-theoretic foundation enables more principled user memory selection that aligns more closely with human selection compared to state-of-the-art methods, and models $400$x bigger. Additionally, we show that memory items selected using RUMS result in better response quality compared to existing approaches, while having up to 95\% reduction in cost.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 22680