Explaining the Plans of Agents via Theory of Mind

Maayan Shvo, Toryn Q. Klassen, Sheila A. McIlraith

Keywords: Epistemic Planning, Theory of Mind, Explanation

TL;DR: We discuss how agents can use their Theory of Mind to resolve discrepancies between their beliefs and the beliefs of other agents regarding plan validity, and showcase the efficacy of our epistemic planning-based approach via a user study.

Abstract: For a plan to achieve some goal – to be valid – a set of sufficient and necessary conditions must hold. In dynamic settings, agents may come to hold false beliefs about these conditions and, by extension, about the validity of their plans or the plans of other agents. Since different agents often believe different things about the world and about the beliefs of other agents, discrepancies may occur between agents’ beliefs about the validity of plans. In this work, we explore how agents can use their Theory in Mind to identify and correct other agents’ beliefs that give rise to discrepancies pertaining to plan validity. We appeal to an epistemic logic framework to allow agents to reason over the nested beliefs of other agents. We realize our approach using epistemic planning and demonstrate how an off-the-shelf epistemic planner can be used to resolve discrepancies regarding plan validity in a number of domains. A study showcases our approach’s ability to resolve misconceptions held by humans pertaining to plan validity.