In this work, we investigate how the explanation of a robot's initially perceived capabilities is based on its surface-level clues and morphology. We explore how explainable robots tie into Robot Theory of Mind (RToM), a term we use to describe how people develop a mental model of a robot. We have developed a web-based platform to collect robot designs that are expected to correspond to mental states. We will train a set of Machine Learning (ML) models focused on feature extraction, validation of desired robot design attributes, and eventually use this as a tool to generate new robot designs targeting designs that provide an initial explanation about the robot's capabilities. We propose a series of neuroscientific studies to iteratively verify the outcomes from the data collection and the ML models training on data provided by the Build-A-Bot platform.

Explainable Robot Design Based on a Robot Theory of Mind: A Web-Based Platform to systematically evaluate Robot Designs

For robotics applications, clear communication between the AI planning system and its human overseer is crucial. Depending on the level of expertise the human has, the availability of live sensor data, and the accuracy of this data, accurate decision-making can be challenging. This paper presents an approach to evaluating a plan's causal structure and a user's decision to remove an action from a plan using domain-specific argumentation schemes. The approach is designed in a modular way and is intended to be used with any PDDL (Planning Domain Definition Language) planning system and easily customizable to its intended application. 

Argumentation for Evaluative Explanations of PDDL Plans

Collaborative robots need to possess the ability to hand objects properly to humans. Earlier studies on robot-to-human handovers have centred around enhancing the human partner's performance and reducing the physical exertion required to grasp the object. Nonetheless, robots exhibiting overly altruistic behaviours may generate protracted and awkward movements that create uncomfortable feelings for humans and affect perceived safety and social acceptance. This paper examines whether applying the cognitive science principle that "humans act coefficiently as a group" in human-robot collaboration - i.e. maximising the benefits for all parties involved simultaneously - leads to a smoother and more natural interaction. Human-robot coefficiency is modelled by online monitoring of human comfort and discomfort indicators and computing robot energy consumption. This score is used by a reinforcement learning problem to adaptively learn the optimal combination of robot interaction parameters to maximise such coefficiency during the task execution. Results demonstrated that by acting coefficiently, the robot accommodated the individual preferences of the majority of participants and enhanced the human perceived comfort.

Maximising Coefficiency of Human-Robot Handovers through Reinforcement Learning

Explicability in AI and robotics is at the centre of interdisciplinary discussions, which involve machine learning, computer science, ethics, and more. Transparency and explicability are generally seen as a need for systems that encompass AI. We offer a different point of view, arguing for re-thinking the necessity of explicability at all costs, citing examples from videogames user-centred design, and existing applications in social robotics.

"Never complain, never explain":  why robots may not have to be explicable after all 

Can robots' observable behavior give human observers insights into their hidden internal processes? This paper examines this idea with the aim of developing a framework for explainability in robotics. This framework constitutes a promising avenue for the development of intuitive human-robot interactions in which people would use the same brain mechanisms on which they rely to understand other humans during social interactions.

Explanation through behavior: a human-inspired framework for explainable robotics

In this work, we investigate the use of semantic attention to explain the performance of a Graph Neural Network (GNN)-based pose estimation model. To validate our approach, we apply semantically-informed perturbations to the input data and correlate the predicted feature importance weights with the accuracy of the model. Graph Deep Learning (GDL) is an emerging field of machine learning for tasks like scene interpretation, as it exploits flexible graph structures to describe complex features and relationships in a very concise format. However, due to the unconventional structure of the graphs, traditional explainability methods used in eXplainable AI (XAI) require further adaptation and thus, graph-specific methods are introduced. Attention is a powerful tool, introduced to estimate the importance of input features in deep learning models. It has been previously used to provide feature-based explanations on the predictions of GNN models. In our proposed work, we exploit graph attention to identify key semantic classes for lidar pointcloud pose estimation. We extend the current attention- based graph explainability methods by investigating the use of attention weights as importance indicators of semantically sorted feature sets by analysing the correlation between attention weights distribution and model accuracy. Our method has shown promising results for post-hoc semantic explanation of graph-based pose estimation.

Towards Semantic Interpretation and Validation of Graph Attention-based Explanations

Multi-Robot Motion Planning (MRMP) is a fundamental problem in robotics and artificial intelligence (AI) where the goal is to calculate trajectories for multiple robots such that every vehicle safely reaches their respective goal when all trajectories are executed simultaneously. One limitation of current MRMP algorithms is their inability to explain their plans to human users. This limitation hinders MRMP algorithms’ use in safety-critical applications, such as air-traffic control, because a human cannot validate the plans. To this end, our study focuses on developing explainable MRMP algorithms that enable a human controller to efficiently validate automatically generated MRMP plans. We base explanations on the simplicity of visual human validation. Specifically, research shows that recognizing line intersections occurs very early in the cognitive process (namely in the primary visual cortex). Thus, MRMP can be explained using a collection of non-intersecting trajectory segments. To this end, we present two newly presented MRMP algorithms capable of automatically generating MRMP plans that can be satisfactorily explained to a human user. 

Explainable Multi-Robot Motion Planning via Segmentation

The crucial role of explanations in making AI safe and trustworthy was not only recognized by the machine learning community but also by roboticists and human–robot interaction researchers. A robot that can explain its actions is supposed to be better perceived by the user, be more reliable, and seem more trustworthy. In collaborative scenarios, explanations are often expected to even improve the team's performance. To test whether a developed explanation-related ability meets these promises, it is essential to rigorously evaluate them. Due to the many aspects of explanations that can be evaluated, and their varying importance in different circumstances, a plethora of evaluation methods are available. In this survey, we provide a comprehensive overview of such methods while discussing features and considerations unique to explanations given during human–robot interactions.

A Survey of Evaluation Methods and Metrics for Explanations in Human–Robot Interaction (HRI)

Current approaches to implement eXplainable Autonomous Robots (XAR) are dominantly based on Reinforcement Learning (RL), which are suitable for modelling and correcting people’s first-order mental state attributions to robots. Our recent findings show that people also rely on attributing second-order beliefs (i.e., beliefs about beliefs) to robots to interpret their behavior. However, robots arguably form and act primarily on first-order beliefs and desires (about things in the environment) and do not have a functional “theory of mind”. Moreover, RL models may be incapable to appropriately address second-order belief attribution errors. This paper aims to open a discussion of what our recent findings on second-order mental state attribution to robots imply for current approaches to XAR.

The challenges of first- and second-order belief reasoning in explainable human-robot interaction

Adoption and deployment of robotic and autonomous systems in industry are currently hindered by the lack of transparency, required for safety and accountability. Methods for providing explanations are needed that are agnostic to the underlying autonomous system and easily updated. Furthermore, different stakeholders with varying levels of expertise, will require different levels of information. In this work, we use surrogate models to provide transparency as to the underlying policies for behaviour activation.  We show that these surrogate models can effectively break down autonomous agents' behaviour into explainable components for use in natural language explanations.

ICRA 2023 Workshop XRo Submissions