Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models
Submission Type: Regular Long Paper
Submission Track: Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.
Submission Track 2: Linguistic Theories, Cognitive Modeling, and Psycholinguistics
Keywords: Pragmatic Reasoning, Rational Speech Act, Quantifier Understanding, Generalized Quantifiers
TL;DR: We crowdsourced dataset for understanding quantifier semantics and propose a pragmatic-based quantifier understanding framework that outperforms literal interpretation of quantifier semantics.
Abstract: Generalized quantifiers (e.g., $\textit{few}$, $\textit{most}$) are used to indicate the proportions predicates satisfy (for example, $\textit{some}$ apples are red). One way to interpret quantifier semantics is to explicitly bind these satisfactions with percentage scopes (e.g., 30%-40% of apples are red). This approach can be helpful for tasks like logic formalization and surface-form quantitative reasoning (Gordon and Schubert, 2010; Roy et al., 2015). However, it remains unclear if recent foundation models (Bommasani et al., 2021) possess this ability due to the absence of direct training signals. To explore this, we introduce QuRe, a crowd-sourced dataset of human-annotated generalized quantifiers in Wikipedia sentences featuring percentage-equipped predicates. We explore quantifier comprehension using PRESQUE, a framework that combines natural language inference and the Rational Speech Acts framework. Experimental results on the HVD dataset (Herbelot and Vecchi, 2015) and QuRe demonstrate PRESQUE's superiority over a literal listener baseline, showing a 20% relative improvement in F1 in predicting percentage scopes for quantifiers, even with no additional training.
Submission Number: 4413
Loading