CLUES: A Benchmark for Learning Classifiers using Natural Language ExplanationsDownload PDF

13 Mar 2022 (modified: 03 Jul 2024)LNLSReaders: Everyone
TL;DR: We introduce a benchmark to learn classifiers, for structured datasets, from natural language explanations.
Abstract: Supervised learning has traditionally focused on inductive learning by observing labeled examples of a task. In contrast, humans have the ability to learn new concepts from language. Here, we explore training classifiers for structured data purely from language. We introduce CLUES, a benchmark for Classifier Learning Using natural language ExplanationS, consisting of a range of classification tasks over structured data along with language supervision in the form of explanations. CLUES consists of 36 real-world and 144 synthetic classification tasks. It contains crowdsourced explanations describing real-world tasks from multiple teachers and programmatically generated explanations for the synthetic tasks. We also develop ExEnt, an entailment-based model that learns classifiers using explanations. ExEnt generalizes up to 18% better (relative) on novel tasks than a baseline that does not use explanations.
Track: Non-Archival (will not appear in proceedings)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 2 code implementations](https://www.catalyzex.com/paper/clues-a-benchmark-for-learning-classifiers/code)
0 Replies

Loading