CAR: Conceptualization-Augmented Reasoner for Zero-Shot Commonsense Question Answering

Weiqi Wang, Tianqing Fang, Wenxuan Ding, Baixuan Xu, Xin Liu, Yangqiu Song, Antoine Bosselut

Submission Type: Regular Long Paper

Submission Track: Commonsense Reasoning

Submission Track 2: Question Answering

Keywords: commonsense reasoning, conceptualization, zero shot, question answering

TL;DR: We propose CAR, a zero-shot commonsense question-answering framework that leverages conceptualization. Experiments demonstrate that CAR surpasses all existing methods, even including GPT3.5 and ChatGPT, and achieves state-of-the-art performance.

Abstract:

The task of zero-shot commonsense question answering evaluates models on their capacity to reason about general scenarios beyond
those presented in specific datasets. Existing approaches for tackling this task leverage external knowledge from CommonSense Knowledge Bases (CSKBs) by pre-training the model on synthetic QA pairs constructed from CSKBs. In these approaches, negative examples (distractors) are formulated by randomly sampling from CSKBs using fairly primitive keyword constraints. However, two bottlenecks limit these approaches: the inherent incompleteness of CSKBs limits the semantic coverage of synthetic QA pairs, and the lack of human annotations makes the sampled negative examples potentially uninformative and contradictory.

To tackle these limitations above, we propose Conceptualization-Augmented Reasoner (CAR), a zero-shot commonsense question-answering framework that fully leverages the power of conceptualization. Specifically, CAR abstracts a commonsense knowledge triple to many higher-level instances, which increases the coverage of the CSKB and expands the ground-truth answer space, reducing the likelihood of selecting false negative distractors. Extensive experiments demonstrate that CAR more robustly generalizes to answering questions about zero-shot
commonsense scenarios than existing methods, including large language models, such as GPT3.5 and ChatGPT. Our code, data, and model checkpoints are available at https://github.com/HKUST-KnowComp/CAR.

Submission Number: 517