Foundations of Symbolic Languages for Model Interpretability

Marcelo Arenas, Daniel Báez, Pablo Barcelo, Jorge Pérez, Bernardo Subercaseaux

Keywords: language, interpretability, logic, complexity, decision trees, explainability

TL;DR: We propose a systematic study of model interpretability by considering a minimalistic symbolic query language tailored for interpretability, studying its complexity of evaluation and presenting a prototype with high-level syntax.

Abstract: Several queries and scores have recently been proposed to explain individual predictions over ML models. Examples include queries based on “anchors”, which are parts of an instance that are sufficient to justify its classification, and “feature-perturbation” scores such as SHAP. Given the need for flexible, reliable, and easy-to-apply interpretability methods for ML models, we foresee the need for developing declarative languages to naturally specify different explainability queries. We do this in a principled way by rooting such a language in a logic called FOIL, which allows for expressing many simple but important explainability queries, and might serve as a core for more expressive interpretability languages. We study the computational complexity of FOIL queries over two classes of ML models often deemed to be easily interpretable: decision trees and more general decision diagrams. Since the number of possible inputs for an ML model is exponential in its dimension, tractability of the FOIL evaluation problem is delicate but can be achieved by either restricting the structure of the models, or the fragment of FOIL being evaluated. We also present a prototype implementation of FOIL wrapped in a high-level declarative language and perform experiments showing that such a language can be used in practice.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/AngrySeal/FOIL-Prototype

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2110.02376/code)