Teach Me to Explain: A Review of Datasets for Explainable Natural Language ProcessingDownload PDF

08 Jun 2021, 00:07 (edited 12 Jan 2022)NeurIPS 2021 Datasets and Benchmarks Track (Round 1)Readers: Everyone
  • Keywords: NLP, explainability, explainable NLP, explainable AI, XAI
  • TL;DR: We identify datasets with 3 classes of textual explanations, organize the literature on annotating each type, identify strengths/shortcomings of existing collection methodologies, and give recommendations for collecting explanations in the future.
  • Abstract: Explainable Natural Language Processing (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future.
  • Supplementary Material: zip
  • URL: https://exnlpdatasets.github.io/
  • Contribution Process Agreement: Yes
  • Dataset Url: https://exnlpdatasets.github.io/
  • Dataset Embargo: N/A
  • License: N/A
  • Author Statement: Yes
8 Replies

Loading