Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions

Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, Yejin Choi

Abstract:

Generics express generalizations about the
world (e.g., birds can fly) that are not universally true (e.g., newborn birds and penguins
cannot fly). Commonsense knowledge bases,
used extensively in NLP, encode some generic
knowledge but rarely enumerate such exceptions and knowing when a generic statement
holds or does not hold true is crucial for developing a comprehensive understanding of
generics. We present a novel framework informed by linguistic theory to generate EXEMPLARS—specific cases when a generic holds
true or false. We generate ∼19k exemplars for
∼650 generics and show that our framework
outperforms a strong GPT-3 baseline by 12.8
precision points. Our analysis highlights the
importance of linguistic theory-based controllability for generating exemplars, the insufficiency of knowledge bases as a source of exemplars, and the challenges exemplars pose
for the task of natural language inference.