ArguBias: Quantifying the Impact of Semantic-Positional Misalignment on Argument Similarity

ACL ARR 2025 May Submission5184 Authors

Abstract: Despite NLP advances, computational approaches for judging argument similarity face a fundamental challenge: semantic-positional dissonance. Embedding models must distinguish between arguments sharing similar linguistic characteristics yet advancing opposing positions, and conversely recognizing when diverse linguistic expression across different cultural, societal, and philosophical contexts convey identical positions. This distinction between content, rhetoric, and position is a complex issue that requires insight from both cognitive science and computational social science. To address this challenge, we introduce ArguBias, a novel framework that systematically identifies, evaluates, and improves similarity judgments for arguments containing cognitive bias structures. First, we introduce the ArguBias Corpus, containing 8,000 annotated argument pairs facilitating the taxonomy of previously unexamined cognitive bias structures in argumentation. This allows us to benchmark 10 state-of-the-art embedding models on their cognitive bias vulnerability. Finally, we demonstrate how minimal fine-tuning on the ArguBias corpus reduces vulnerability of embedding models to cognitive bias structures by up to 11.6pp. Simultaneous gains of 7.1% and 5.4% on argument similarity benchmarks BWS and AFS indicate generalizability and improvement of fundamental semantic understanding beyond domain-specific applications.

Paper Type: Long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: argument mining, argument schemes and reasoning, rhetoric and framing, stance detection, style analysis

Contribution Types: Model analysis & interpretability, Data resources, Data analysis

Languages Studied: English

Submission Number: 5184