Analytical Reasoning of Text

Anonymous

Paper Link: https://openreview.net/forum?id=1VhssoBf8HL

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: Analytical reasoning is an essential and challenging task that requires a system to analyze a scenario involving a set of particular circumstances and perform reasoning over it to make conclusions. However, current neural models with implicit reasoning ability struggle to solve this task. In this paper, we study the challenge of analytical reasoning of text and collect a new dataset consisting of questions from the Law School Admission Test from 1991 to 2016. We analyze what knowledge understanding and reasoning abilities are required to do well on this task, and present an approach dubbed ARM. It extracts knowledge such as participants and facts from the context. Such knowledge are applied to an inference engine to deduce legitimate solutions for drawing conclusions. In our experiments, we find that ubiquitous pre-trained models struggle to deal with this task as their performance is close to random guess. Results show that ARM outperforms pre-trained models significantly. Moreover, we demonstrate that ARM has better explicit interpretable reasoning ability.

Copyright Consent Signature (type Name Or NA If Not Transferrable): Wanjun Zhong

Copyright Consent Name And Address: Sun Yat-Sen University: No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China

Presentation Mode: This paper will be presented virtually

Virtual Presentation Timezone: UTC+8