Debatrix: Advancing Automatic Debate Adjudication with Chronological and Multi-dimensional Analysis

Anonymous

Abstract: How can we construct an automated debate judge to assist with evaluating an extensive, fervent, multi-turn debate? This task is challenging, as judging a debate involves grappling with lengthy texts, intricate argument relationships, and multi-dimensional assessments, while current research mainly focuses on short dialogues, rarely touching upon the evaluation of an entire debate.In this paper, by leveraging Large Language Models (LLMs), we propose Debatrix, which makes the analysis and assessment of multi-turn debates more aligned with majority preferences. Specifically, Debatrix features a horizontal chronological workflow and a vertical multi-dimensional evaluation collaboration.To align with real-world debate scenarios, we introduced DebateArt and DebateCompetition benchmarks, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation.

Paper Type: long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English