As planning and autonomy in general become increasingly deployed on board spacecraft, missions will face a paradigm shift in how ground operations teams command and interact with the spacecraft: moving from specifying timed sequences of commands to high-level goals that on-board autonomy will elaborate based on the spacecraft’s state and sensed environment. It will become increasingly difficult for operators to predict a mission's outcome as autonomous spacecraft venture into deeper space, react to unknown conditions, and face stronger communication constraints. However, data from simulated autonomous missions can be analyzed and leveraged, allowing operators to make informed decisions when selecting mission parameters and high-level goals. To this end, our paper presents a framework that gains insights from simulation data in order to help operators of autonomous spacecraft missions to predict, explain, and search for specific outcomes given a set of high-level goals. We show and discuss how our approach can help operators to better understand predictions, explore options and make informed decisions. To this end, we describe a case study that builds upon previous work on simulated autonomous spacecraft missions to the Neptune-Triton system where the spacecraft uses an automated planning and execution framework to make onboard decisions.

Outcome Prediction and Explainability for Mission Operations of Autonomous Spacecraft

Within the field of automated planning, two areas of study are planning with preferences and epistemic planning. Planning with preferences involves generating plans that optimize for properties of the plan instead of, or in addition to, trying to reach a fixed goal. Epistemic planning allows for planning over the knowledge or belief states of one or more agents for the purpose of achieving epistemic goals (where agents have particular states of knowledge or belief). In this paper we motivate and explore the task of planning with epistemic preferences, proposing a method by which existing automated planning techniques can be combined for this purpose. Epistemic preferences may better allow for representing what humans want, and have benefits for AI safety.

Towards Human-Aware AI via Planning with Epistemic Preferences

Deep learning is increasingly used to learn policies for planning problems. However, policies represented by neural networks are difficult to interpret, verify and trust. Existing formal approaches to post-hoc explanations provide concise reasons for a single decision made by an ML model. However, understanding planning policies requires explaining sequences of decisions. In this paper,  we formulate the problem of finding explanations for the sequence of decisions recommended by a learnt policy in a given state. We show that, under certain assumptions, a minimal explanation for a sequence can be computed by solving a  number of single decision explanation problems which is linear in the length of the sequence. We present experimental results of our implementation of this approach for ASNets policies for classical planning domains.

Formal Explanations of Neural Network Policies for Planning

The recognition of activities performed by humans is crucial in human-robot interaction. However, assuming humans always follow rational behaviour in executing activities may not be accurate since individual preferences influence their decision-making. This paper proposes a method for learning human behaviour that involves capturing how humans select actions to solve problems. This behaviour is represented by a Relational Decision Tree. We define two sets of features that can be automatically extracted from the planning domain. A behaviour library is created and used to identify the behaviour followed by a person when executing a plan in a new situation. This approach allows to anticipate the person’s needs and act accordingly. The method was tested in three different domains, showing its validity.

Learning and Recognizing Human Behaviour with Relational Decision Trees

Oversubscription planning (OSP) is a planning formalism that can deal with planning scenarios where not all goals can be achieved. If the global optimization goal is not fixed, an iterative process in which users refine their preferences based on the sample plans is suitable. To help the users, the planning systems should be able to provide answers to questions such as ``Why is goal $g$ not satisfied by the sample
plan?''

In this paper, we focus on explaining the behavior of a given black-box policy under the aforementioned planning scenario. That is, we assume that sample plans are provided by a state-dependent deterministic policy $\phi$ (in our case a neural network
policy), and we try to automatically answer the question ``Why is the goal $g$ not satisfied by $\phi$?'' by providing information how much 
the policy $\phi$ would need to diverge from its decision in order to satisfy the goal $g$. Moreover, we also provide information in which state and how the policy should diverge. To this end, we extended action policies based on action schema networks to
support OSP tasks and design an algorithm that is able to provide the desired explanations. We evaluate the performance of the proposed algorithm and provide a case study with sample explanations.

Action Policy Explanations in Oversubscription Planning

In this paper, we introduce DR-HAI (Dialectical Reconciliation in Human-AI Interactions), a novel game-theoretic framework designed to extend model reconciliation approaches for enhanced human-AI interaction. By adopting a multi-shot reconciliation paradigm and not assuming a-priori knowledge of the human user's model, DR-HAI enables interactive dialogues to address knowledge discrepancies between explainee and explainer agents. We provide formal operational semantics for DR-HAI using logic-based argumentation and offer theoretical guarantees regarding the framework's termination and success. Furthermore, we conduct a human-user study that compares DR-HAI to single-shot reconciliation approaches, demonstrating the efficacy of our framework in improving users' understanding of AI decisions in tasks characterized by substantial knowledge asymmetry. Our findings suggest that DR-HAI offers a promising direction for fostering effective human-AI interactions.


DR-HAI: Argumentation-based Dialectical Reconciliation in Human-AI Interactions

The ability to easily and unambiguously specify a goal to a planner is fundamental to human-AI collaboration and knowledge discovery. Recently, deep reinforcement learning has been used to train deep neural networks (DNNs) as heuristic functions for planning problems. While DNNs can be powerful function approximators that, combined with reinforcement learning, require little to no domain-specific knowledge to learn, there is no formal way to specify goals to DNNs. We introduce a method of training DNN heuristic functions to estimate the distance between a given state and a goal, where a goal is represented as a set of atoms in first-order logic. We then use answer set programming to specify goals, where a set of atoms representing a goal is obtained from the stable model of an answer set program. The DNN heuristic function is then combined with search to reach goals. In our experiments with the Rubik's cube and Sokoban, we show that we can specify and reach a variety of different goals without any need to re-train the DNN. Furthermore, since the specification language is first-order logic, one can specify a goal without having to know what states meet that specification, beforehand. Therefore, our approach can also be used to discover states that meet a given specification.

Specifying Goals to Deep Neural Networks with Answer Set Programming

A key component for building trust in automated systems is improved transparency into the decisions made by the automation and the reasons behind those decisions. Current models of transparency, however, assume that the operator has the time and mental capacity to digest the transparency information the moment it is provided. Often times, this is exactly the moment where the operator is most overloaded, which is why they delegated to automation in the first place. This paper presents the idea of Transparency Information Pacts, or TIPs, as a way to formally represent transparency information and better allow the information to be requested and conveyed back to the human when they are best able to use it. TIPs build off the idea of Lifecycle Transparency presented in (Miller 2021), which encourages the use of other mission phases--pre-mission planning and post-mission debrief--for conveying transparency information where appropriate. Here we present multiple types of TIPs, how they are structured and used, along with illustrative examples.

TIPs: Transparency Information Pacts

Understanding problem solving or planning has been a shared challenge for both AI and cognitive science since the birth of both fields. We explore the extent to which modern planners from the field of AI can account for human performance on the Tower of London (TOL) task, a close relative of the Tower of Hanoi problem that has been extensively studied by psychologists. We characterize the task using the Planning Domain Definition Language (PDDL) and evaluate an adaptive online planner and a family of well-known planners, including online planners, optimal planners and satisficing planners. Each planner is evaluated based on its ability to predict the actions and planning times of participants in a new behavioral experiment. Our results suggest that participants use a range of strategies but that an adaptive lookahead planner provides the best overall account of both human actions and human planning times. This finding is consistent with the view that humans differ from standard AI planners by integrating a mechanism for evidence accumulation.

Bridging the Gap Between Human and Automated Planning on the Tower of London Task

Although automated planning and scheduling systems based on optimization models are increasingly being adopted into socially responsible tasks, the derived plan is often counterintuitive under complicated considerations. Users will claim the right to know the reason for “Why did the optimal plan include something or not include something else (that I would have chosen)?” Explanations of constraints and parameters that cause the unexpected plan derivation can play an important role in building trust between users and the scheduling system. However, existing approaches require an assumption of a specific problem setting, and have not addressed quantitative analysis for multiple types of factors. In this paper, we propose a general explanation framework to quantitatively evaluate the effect of constraints and parameters on the plan derivation by applying the concept of Shapley values, which satisfy the desirable axioms for explanations. The coalitional game based on optimization models is formulated to calculate the contributions of these factors to the fulfillment of values or conditions in which users are interested. Through numerical experiments of the typical personnel assignment problem, we show that our framework can identify the major causes efficiently under various parameter settings and provide directly understandable explanations compared to the basic contrastive explanations.

ICAPS 2023 Workshop HAXP Submissions