Generating High-Quality Explanations for Navigation in Partially-Revealed Environments

Gregory J. Stein

Keywords: Explainable Planning, Navigation Under Uncertainty, Explainable Artificial Intelligence (XAI)

Abstract: We present an approach for generating natural language explanations of high-level behavior of autonomous agents navigating in partially-revealed environments. Our counterfactual explanations communicate changes to interpratable statistics of the belief (e.g., the likelihood an exploratory action will reach the unseen goal) that are estimated from visual input via a deep neural network and used (via a Bellman equation variant) to inform planning far into the future. Additionally, our novel training procedure mimics explanation generation, allowing us to use planning performance as an objective measure of explanation quality. Simulated experiments validate that our explanations are both high quality and can be used in interventions to directly correct bad behavior; agents trained via our training-by-explaining procedure achieve 9.1% lower average cost than a non-learned baseline (12.7% after interventions) in environments derived from real-world floor plans.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: We generate explanations of a robot agent's behavior as it navigates through a partially-revealed environment, expressed in terms of changes to its predictions about what lies in unseen space.

Code: https://github.com/RAIL-group/xai-nav-under-uncertainty-neurips2021