Conversational Goal-Conflict Explanations in Planning via Multi-Agent LLMs

Guilhem Fouilhé, Rebecca Eifler, Sylvie Thiebaux, Nicholas Asher

Keywords: planning, explainable AI, LLM, multi-agent, user interface

TL;DR: We formalize and implement an LLM-based conversational explanation interface for planning.

Abstract: When automating plan generation for a real-world sequential decision problem, the goal is often not to replace the human planner, but to facilitate the tedious work. In an iterative process, the human’s role is to guide the planner according to their preferences and experience. Explanations that respond to users’ questions are crucial to increase trust in the system and improve understanding of the example solutions. To enable natural interaction with such a system, we present an explanation framework agnostic architecture for interactive natural language explanations that enables user and context dependent interactions. We propose conversational interfaces based on Large Language Models (LLMs) and instantiate the explanation framework with goal-conflict explanations. As a basis for future evaluation, we provide a tool for domain experts that implements our interactive natural language explanation architecture.

Submission Number: 27