An Informal Logic LLM-Based Argumentation Framework

Paulo Pirozelli, Douglas Aldred, Victor Hugo Nascimento Rocha, Fabio Cozman

Track: long paper (up to 10 pages)

Keywords: Informal Logic, Argumentation, LLM, Argument Mining, Argument Evaluation

Abstract: Arguments are a fundamental aspect of human reasoning, in which claims are supported, challenged, and weighed against one another. We present a system that uses large language model (LLM)-based modules to reconstruct arguments from natural language text in accordance with Informal Logic (IL). The system follows a multi-step pipeline that progressively identifies argumentative components, selects relevant elements, and uncovers their logical relations. These elements are represented as a directed acyclic graph, using two component types (premises and conclusions) and three relation types (support, attack, and undercut). We conduct two complementary experiments to evaluate the system. First, we perform a manual evaluation on a dataset of arguments drawn from an IL textbook, enabling a detailed analysis of the system’s ability to capture IL-style argument structures. Second, we carry out a quantitative evaluation on benchmark datasets, allowing comparison with prior work by mapping our outputs to established annotation schemes. Results show that the system is capable of recovering a range of IL-style arguments and, when adapted to different annotation schemes, achieves reasonable performance across benchmark datasets, highlighting the potential of LLM-based pipelines for scalable argument reconstruction.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 187