Embedding‑to‑Prefix: Continual Personalization with Large Language Models

Bernd Huber, Ghazal Fazelnia, Andreas Damianou, Sebastian Peleato, Maksym Lefarov, Praveen Chandar, Marco De Nadai, Mounia Lalmas, Paul N. Bennett

Keywords: Large Language Models, Personalization, Prefix Tuning, User Embeddings, Natural Language Generation

TL;DR: To continually personalize a frozen LLM, our method (E2P) maps dynamic user embeddings to a single soft prefix, boosting recommendation engagement by 12.9%.

Abstract: Large language models (LLMs) excel at generating contextually relevant content, but their static nature prevents adaptation to dynamic, evolving user preferences. Yet capturing the full spectrum of user preferences that evolve over time remains challenging. Existing methods for capturing evolving user preferences and taste profiles often depend on fine-tuning or token-intensive prompting, which typically require significant effort or computational expense. We propose Embedding-to-Prefix (E2P), a parameter-efficient adaptation that injects pre-computed user embeddings into an LLM through a learned projection to a single soft token prefix. This enables effective personalization while keeping the backbone model frozen, providing a compatible and cost-effective mechanism for continual model updates. We evaluate E2P in two large-scale production settings where user embeddings are dynamically updated: music playlist generation and podcast recommendation. Our results show that E2P achieves a 12.9% improvement in user engagement for music recommendation and provides complementary personalization signal in podcast recommendation.

Serve As Reviewer: ~Bernd_Huber1

Submission Number: 6