JurisMA-CQAD: A Multi-Agent Framework and Dataset for Legal Consultation Question Answering

ACL ARR 2025 May Submission6182 Authors

Abstract: Legal consultation question answering (Legal CQA) presents unique challenges that differ substantially from traditional legal QA tasks, including high contextual dependency, multi-stage reasoning, and a lack of large-scale annotated datasets. To address these issues, we propose JURISMA, a modular multi-agent collaborative framework to decompose complex legal queries into interpretable subtasks. Our system integrates a structured legal element graph for semantic grounding, a Draft Agent for initial opinion generation, and a Manager Agent to dynamically coordinate refinement through auxiliary agents such as FormatCheck and LawSearch. To facilitate training and evaluation, we construct JURISCQAD, a novel dataset comprising over 43,000 real-world Chinese legal consultations, annotated with both positive and adversarial responses under expert supervision. Experiments on the LawBench benchmark demonstrate that our approach significantly outperforms state-of-the-art general and legal-domain LLMs across multiple lexical and semantic metrics.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: legal NLP,interactive and collaborative generation,dataset construction,LLM/AI agents,

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Publicly available software and/or pre-trained models, Data resources, Data analysis

Languages Studied: Chinese,English

Submission Number: 6182