JECC: Commonsense Reasoning Tasks Derived fromInteractive Fictions

Mo Yu, Xiaoxiao Guo, Yufei Feng, Yi Gu, Xiaodan Zhu, Michael Greenspan, Murray Campbell, Chuang Gan

Keywords: text games, commonsense reasoning, text entailment, natural language processing

TL;DR: We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs; and conduct experiments and human study to verify its challenges.

Abstract: Commonsense reasoning simulates the human ability to make presumptions about our physical world, and it is an essential cornerstone in building general AI systems. We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs as human players demonstrate plentiful and diverse commonsense reasoning. The new dataset provides a natural mixture of various reasoning types and requires multi-hop reasoning. Moreover, the IF game-based construction procedure requires much less human interventions than previous ones. Experiments show that the introduced dataset is challenging to previous machine reading models with a significant 20% performance gap compared to human experts.

URL: https://github.com/Gorov/zucc