Learning from Implicit User Feedback, Demographic Information and User Emotions in Task-Oriented Document-Grounded Dialogues

Anonymous

Abstract: Trustworthiness, interaction quality and empathy have a great influence on whether users accept a dialogue system. To address this, recent works on open-domain dialogues suggest to learn from implicit user feedback or to consider demographic information and user emotions in response generation to improve generation accuracy and user engagement. However, for task-oriented and document-grounded dialogue systems, task completion and factual consistency of the generated responses are almost more important. The impact of such data on these quality criteria is not yet known. To address this gap, we (1) introduce \ourdata, the first English task-oriented document-grounded dialogue dataset annotated with implicit user feedback, demographic information and user emotions, and (2) investigate the impact of including such data on task completion, and the factual consistency of responses generated by Flan-T5, GPT-2, and Llama 2. Our results show a particularly positive impact on task completion and factual consistency, and that responses generated by models trained with implicit user feedback are preferred by human users.

Paper Type: long

Research Area: Dialogue and Interactive Systems

Contribution Types: Data resources

Languages Studied: English