Max Explainability Score with Confidence Interval (MES-CI): A Quantitative Metric for Interpretability in Knowledge Graph-Based Recommender System

Neeraj Tiwary, Shahrul Azman Mohd Noah

Keywords: - Explainable AI, Recommender Systems, Knowledge Graphs, Confidence Interval, Explainability Evaluation Metrics, Reinforcement Learning

Abstract: Knowledge graph-based recommender systems (KGRS) utilize structured semantic relationships to generate personalized and interpretable recommendations, leveraging the inherent connectivity within knowledge graphs to enhance transparency. While KGRS offer significant advantages in explainability, quantifying the reliability and impact of these explanations remains challenging due to the complexity of underlying models and the multiple pathways that influence recommendation outcomes. This paper critically analyzes existing evaluation metrics for explainability in KGRS, identifying their limitations and advocating for a balanced framework that integrates interpretability with predictive accuracy. This research builds upon the existing Max Explainability Score (MES) by introducing an enhanced scoring mechanism, the Max Explainability Score with Confidence Interval (MES-CI). MES-CI overcomes the limitations of evaluating the explainability of generated recommendations using a single-point score by providing a more comprehensive and balanced assessment. It incorporates confidence intervals alongside confidence score percentages, offering a clearer representation of explainability reliability. Furthermore, the applicability of this refined metric is examined across multiple datasets, with case studies demonstrating its effectiveness in improving transparency and user trust in AI-driven recommendation systems.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 18329