Task-aware Contrastive Mixture of Experts for Quadruple Extraction in Conversations with Code-like Replies and Non-opinion Detection

ACL ARR 2025 February Submission2289 Authors

Abstract: Accurate extraction of quadruples from dialogues is essential for advancing natural language understanding and supporting applications such as dialogue systems and knowledge graph construction. Applying Large Language Models (LLMs) for this specific task presents two primary challenges: the accurate extraction of multiple elements and the understanding of complex dialogue reply structure. To tackle these issues, we propose a novel LLM-based multi-task approach, named $\textbf{T}$ask-$\textbf{a}$ware $\textbf{Co}$ntrastive $\textbf{M}$ixture $\textbf{o}$f $\textbf{E}$xperts ($\textbf{TaCoMoE}$), to tackle the DiaASQ task by integrating expert-level contrastive loss within task-oriented mixture of experts layer. TaCoMoE minimizes the distance between the representations of the same expert in the semantic space while maximizing the distance between the representations of different experts to efficiently learn representations of different task samples. Additionally, we design a Graph-Centric Dialogue Structuring strategy for representing dialogue reply structure and perform non-opinion utterances detection to enhance the performance of quadruple extraction. Extensive experiments are conducted on the DiaASQ dataset, demonstrating that our method significantly outperforms existing parameter-efficient fine-tuning techniques in terms of both accuracy and computational efficiency. The code will be released soon.

Paper Type: Long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: Large Language Model,Mixture of Experts,quadruple extraction in conversations,contrastive learning

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English, Chinese

Submission Number: 2289