Learning to Scaffold: Optimizing Model Explanations for Teaching

Patrick Fernandes, Marcos Vinicius Treviso, Danish Pruthi, Andre Martins, Graham Neubig

Keywords: explainable ai, interpretability, meta-learning, attention

Abstract: Modern machine learning models are opaque, and as a result there is a burgeoning academic subfield on methods that explain these models' behavior. However, what is the precise goal of providing such explanations, and how can we demonstrate that explanations achieve this goal? Some research argues that explanations should help teach a student (either human or machine) to simulate the model being explained, and that the quality of explanations can be measured by the simulation accuracy of students on unexplained examples. In this work, leveraging meta-learning techniques, we extend this idea to improve the quality of the explanations themselves, specifically by optimizing explanations such that student models more effectively learn to simulate the original model. We train models on three natural language processing and computer vision tasks, and find that students trained with explanations extracted with our framework are able to simulate the teacher significantly more effectively than ones produced with previous methods. Through human annotations and a user study, we further find that these learned explanations more closely align with how humans would explain the required decisions in these tasks. Our code is available at https://github.com/coderpat/learning-scaffold.

TL;DR: Automatically learn to explain the decision of models by maximizing how well students simulate said model

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:2204.10810/code)