Exploring the Relationship Between Feature Attribution Methods and Model Performance

Priscylla Silva, Claudio Silva, Luis Gustavo Nonato

Track: Responsible AI for Education (Day 2)

Paper Length: long-paper (6 pages + references)

Keywords: Explainable Artificial Intelligence, Educational Predictions, Student Success, Explanation Methods, Model Performance, Feature Importance, Correlation Analysis

TL;DR: Our study highlights the relationship between the model's performance and the agreement level among explanation methods used to generate explanations for the model's predictions.

Abstract: Machine learning and deep learning models are pivotal in educational contexts, particularly in predicting student success. Despite their widespread application, a significant gap persists in comprehending the factors influencing these models' predictions, especially in explainability within education. This work addresses this gap by employing nine distinct explanation methods and conducting a comprehensive analysis to explore the correlation between the agreement among these methods in generating explanations and the predictive model's performance. Applying Spearman's correlation, our findings reveal a very strong correlation between the model's performance and the level of agreement observed among the explanation methods.

Submission Number: 55