Controlled Evaluation of Explanations: What Might Have Influenced Your Model Explanation Efficacy Evaluation?

Mimansa Jaiswal

Abstract: End users affect explanation efficacy. NLP papers overlook other factors. This paper examines how utiltiy of saliency-based explanations change with controlled variables. We aim to provide a standardized list of variables to evaluate and show how SoTA algorithms rank differently when controlling for evaluation criteria.