T-REX: Trustworthy and Reliable EXplainability Framework

Michal Zakrzewski, Mariana Temnyk, Taras Rumezhak, Volodymyr Karpiv

Abstract: The academic and commercial interest in the explainability of neural networks increases as they transition from research to production. The need to develop trust and understanding of the model’s performance in critical domains arises in spite of the lack of automated tools for trustworthiness assessment. One can observe that current research is not rich in unified metrics to compare the interpretability of models directly instead of a manual visual comparison of class activation maps. Contrary, our research proposes a new evaluation analytical framework\(^1\) to assess automatically and benchmark computer vision methods on their explainability based on direct metrics and apply them to different use cases in cloud monitoring tools. To achieve this, we utilize human saliency map datasets with annotations that help to compare the activation map values and ground truth. We also utilize human-logic-based metrics to evaluate explainability without additional labels. With the proposed framework, we validate explainability and trustworthiness hypotheses on model performance, explanation methods, and robustness. Moreover, we propose an applied usage of the explainability metrics to solve real tasks during continuous model deployment and monitoring with the help of our framework.\(^1\)https://github.com/TREXSubmission/T-REX

External IDs:dblp:conf/icaisc/ZakrzewskiTRK24