Expectation Confirmation Preference Optimization for Multi-Turn Conversational Recommendation Agent

ACL ARR 2025 February Submission3521 Authors

Abstract:

Recent advancements in Large Language Models (LLMs) have significantly propelled the development of Conversational Recommendation Agents (CRAs). However, these agents often generate short-sighted responses that fail to sustain user guidance and meet expectations.
Although preference optimization has proven effective in aligning LLMs with user expectations, it remains costly and performs poorly in multi-turn dialogue.
To address this challenge, we introduce a novel multi-turn preference optimization (MTPO) paradigm **ECPO**, which leverages Expectation Confirmation Theory to explicitly model the evolution of user satisfaction throughout multi-turn dialogues, uncovering the underlying causes of dissatisfaction.
These causes can be utilized to support targeted optimization of unsatisfactory responses, thereby achieving turn-level preference optimization.
ECPO eliminates the significant sampling overhead of existing MTPO methods while ensuring the optimization process drives meaningful improvements.
To support ECPO, we also introduce an LLM-based user simulator, **AILO**, to simulate user feedback and expectation confirmation during conversational recommendations. Experimental results show that ECPO significantly enhances CRA's interaction capabilities, offering notable improvements in both efficiency and effectiveness over existing MTPO methods.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: task-oriented; applications; fine-tuning; conversational modeling

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 3521