Annotating Implicit Reasoning in Arguments with Causal Links

Anonymous

Keywords: Causality, Implicit reasoning, commonsense, argumentation knowledge, structured representation of reasoning

TL;DR: Designing an annotation process to crowdsource Implicit Reasoning in Arguments with Causal Links

Abstract:

Most of the existing work that focus on the identification of implicit knowledge in arguments generally represent implicit knowledge in the form of commonsense or factual knowledge.
However, such knowledge is not sufficient to understand the implicit reasoning link between individual argumentative components (i.e., claim and premise).
In this work, we focus on identifying the implicit knowledge in the form of argumentation knowledge which can help in understanding the reasoning link in arguments.
Being inspired by the Argument from Consequences scheme, we propose a semi-structured template to represent such argumentation knowledge that explicates the implicit reasoning in arguments via causality.
We create a novel two-phase annotation process with simplified guidelines and show how to collect and filter high quality implicit reasonings via crowdsourcing.
We find substantial inter-annotator agreement for quality evaluation between experts, but find evidence that casts a few questions on the feasibility of collecting high quality semi-structured implicit reasoning through our crowdsourcing process.
We release our materials (i.e., crowdsourcing guidelines and collected implicit reasonings) to facilitate further research towards the structured representation of argumentation knowledge.

Submission Type: Non-Archival Full Paper