SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoning

Yunxiang Zhang, Xiaojun Wan

Keywords: generative commonsense reasoning, generative language model

TL;DR: We introduce a generative commonsense reasoning dataset to evaluate how well machines reason under contrastive geographical and temporal contexts.

Abstract: Recently, commonsense reasoning in text generation has attracted much attention. Generative commonsense reasoning is the task that requires machines, given a group of keywords, to compose a single coherent sentence with commonsense plausibility. While existing datasets targeting generative commonsense reasoning focus on everyday scenarios, it is unclear how well machines reason under specific geographical and temporal contexts. We formalize this challenging task as SituatedGen, where machines with commonsense should generate a pair of contrastive sentences given a group of keywords including geographical or temporal entities. We introduce a corresponding English dataset consisting of 8,268 contrastive sentence pairs, which are built upon several existing commonsense reasoning benchmarks with minimal manual labor. Experiments show that state-of-the-art generative language models struggle to generate sentences with commonsense plausibility and still lag far behind human performance. Our dataset is publicly available at https://github.com/yunx-z/situated_gen.

Submission Number: 508