Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning

Xiao Yu, Maximillian Chen, Zhou Yu

Submission Type: Regular Short Paper

Submission Track: Dialogue and Interactive Systems

Keywords: Prompting, MCTS, Dialogue Policy Planning

TL;DR: We introduce GDP-Zero, an approach combining Open-Loop MCTS and prompting large language models, to perform goal-oriented dialogue policy planning without any model training.

Abstract: Planning for goal-oriented dialogue often requires simulating future dialogue interactions and estimating task progress. Many approaches thus consider training neural networks to perform look-ahead search algorithms such as A* search and Monte Carlo Tree Search (MCTS). However, this training often require abundant annotated data, which creates challenges when faced with noisy annotations or low-resource settings. We introduce GDP-Zero, an approach using Open-Loop MCTS to perform goal-oriented dialogue policy planning without any model training. GDP-Zero prompts a large language model to act as a policy prior, value function, user simulator, and system model during the tree search. We evaluate GDP-Zero on the goal-oriented task PersuasionForGood, and find that its responses are preferred over ChatGPT up to 59.32% of the time, and are rated more persuasive than ChatGPT during interactive evaluations.

Submission Number: 1262