Review Texts as Data-Efficient Signals for LLM-based Personalization

ACL ARR 2025 May Submission1971 Authors

Abstract:

Off-the-shelf large language models (LLMs) have been showing promising performance in personalization based on user preference.
However, previous studies mainly discuss using numeric signals such as scores, which require many data points for satisfactory performance.
Some systems based on fine-tuned LLMs have achieved reasonable performance by using review texts as additional information, but their use with off-the-shelf LLMs is underexplored.
This work aims to clarify the effects of review texts on off-the-shelf LLM–based personalization from various perspectives.
By comparing multiple prompt formats with different in-context information, we show that per-item review texts can improve the user rating prediction performance by off-the-shelf LLMs across different datasets and models, even with a few data points.
We also find that instructing LLMs to write expected reviews can improve the performance, while general prompt engineering techniques such as zero-shot chain-of-thought can result in a worse performance.
These results open the possibility of LLM-based personalization systems with fewer required data points.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Recommendation, Personalization, User Rating Prediction, User Modeling, Prompt Engineering

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 1971