Echoes of Agreement: Argument driven opinion shifts in Large Language models

ACL ARR 2025 May Submission7801 Authors

Abstract: There have been numerous studies conducting bias evaluation of LLMs on political topics. However, how positions in model outputs change in presence of arguments towards those positions remains underexplored. This is crucial for understanding how robust positions in model outputs and the bias evaluations are. To that end, through our experiments we find that the presence or absence of supporting or refuting arguments towards a particular claim, can affect the nature of responses in single and multi-turn setting. This can have a downstream impact on evaluation of political biases and corresponding mitigation strategies.

Paper Type: Short

Research Area: Computational Social Science and Cultural Analytics

Research Area Keywords: Model analysis & interpretability,Human-Centered NLP, Dialogue and Interactive Systems,Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data analysis

Languages Studied: English

Submission Number: 7801