Aligning Recommendation Explanations to User Preferences Using LLMs Fine-Tuned by Reinforcement Learning with AI Feedback

Julien Albert, Martin Balfroid, Lluc Bono Rosselló, Arunav Das, Lucile Dierckx, Yanni Sun

Track: Type E (Late-Breaking Abstracts)

Keywords: recommender systems, large language models, reinforcement learning from AI feedback, explainable recommendation

Abstract: We investigate aligning a small language model to generate helpful recommendation explanations with limited access to human evaluators. To improve user satisfaction without requiring extensive human evaluation, we explore the use of reinforcement learning with AI feedback. We conduct both online and offline preliminary evaluations to compare the alignment of fine-tuned small language models against their teacher and their base version. Although our online evaluation was premature, the offline analysis revealed promising directions.

Submission Number: 88