Large Language Models Excel at Zero-Shot Conversation Disentanglement

ACL ARR 2025 February Submission235 Authors

Abstract: Disentangling overlapping conversations in multi-party communication is a foundational challenge in natural language processing. Existing state-of-the-art approaches leverage encoder-based language models, often requiring extensive training data and complex feature engineering. In this work, we explore the capabilities of large language models (LLMs) in conversation disentanglement using zero-shot prompting. We propose two simple, principled prompting schemes for conversation disentanglement, along with a self-critic technique for refining results. Testing on the Ubuntu IRC and Movie Dialogue datasets, our methods surpass previous state-of-the-art performance without requiring model fine-tuning. Comparative analysis with human annotators suggests that LLMs perform comparably to humans, but further work is required to uniformly outperform the median annotator on all metrics.

Paper Type: Short

Research Area: Dialogue and Interactive Systems

Research Area Keywords: dialogue state tracking, conversational modeling

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 235