Safe Explicable Planning

Published: 12 Feb 2024, Last Modified: 06 Mar 2024ICAPS 2024EveryoneRevisionsBibTeXCC BY 4.0
Keywords: Human-aware Planning and Scheduling
TL;DR: This paper introduces a planning formulation to find a human expected policy such that the safety bounds in the agent's model are satisfied.
Abstract: Human expectations stem from their knowledge about the others and the world. Where human-AI interaction is concerned, such knowledge may be inconsistent with the ground truth, resulting in the AI agent not meeting its expectations and degraded team performance. Explicable planning was previously introduced as a novel planning approach to reconciling human expectations and the agent’s optimal behavior for more interpretable decision-making. One critical issue that remains unaddressed is safety in explicable planning since it can lead to explicable behaviors that are unsafe. We propose Safe Explicable Planning (SEP) to extend the prior work to support the specification of a safety bound. The objective of SEP is to search for behaviors that are close to the human’s expectations while satisfying the bound on the agent’s return, the safety criterion chosen in this work. We show that the problem generalizes the consideration of multiple objectives to multiple models and our formulation introduces a Pareto set. Under such a formulation, we propose a novel exact method that returns the Pareto set of safe explicable policies, a more efficient greedy method that returns one of the Pareto optimal policies, and approximate solutions for them based on the aggregation of states to further scalability. Formal proofs are provided to validate the desired theoretical properties of the exact and greedy methods. We evaluate our methods both in simulation and with physical robot experiments. Results confirm the validity and efficacy of our methods for safe explicable planning.
Primary Keywords: Human-aware Planning and Scheduling
Category: Long
Student: Graduate
Supplemtary Material: pdf
Submission Number: 86