Relations Between Explainability, Evaluation and Trust in AI-Based Information Fusion Systems

Gregor Pavlin, Johan Pieter de Villiers, Jürgen Ziegler, Anne-Laure Jousselme, Paulo C. G. Costa, Kathryn B. Laskey, Alta de Waal, Erik Blasch, Lennard Jansen

Abstract: Explainability is generally considered an important means to gain trust in complex automated decision support systems. Different types of explainability of processes and models used in a complex information fusion solution based on Artificial Intelligence (AI) are relevant throughout its life-cycle, i.e. during the system development as well as its deployment. However, it is often difficult to understand the real value of explainability in specific cases. To study the impact of explainability on trust, there is a need to emphasize the trust building processes, especially various types of evaluations supporting trust assessment. The paper emphasizes that the value of explainability is as an enabler of certain types of evaluations leading to improved trust in automated solutions. A conceptual model brings together different types of explainability, evaluations, and operational conditions along with human factors influencing the trust in automated systems. The introduced model describes the types of possible evaluations and related explainability at different stages of life cycles of AI-based information fusion solutions. This enables adaptation of life cycles, such that the trust assessment is facilitated. The concepts are illustrated with the help of examples using different modelling and processing techniques.