Should multiple defendants and charges be treated separately in legal judgment prediction: An exploratory study and dataset
Abstract: Legal judgment prediction offers a compelling method to aid legal practitioners and researchers.
However, the research question remains relatively under-explored:
Should multiple defendants and charges be treated separately in Legal judgment prediction?
To address this, we introduce a new dataset namely multi-person multi-charge prediction MPMCP, and seek the answer by evaluating the performance of several prevailing legal \acp{LLM} on four practical legal judgment scenarios:
(S1) single defendant with a single charge,
(S2) single defendant with multiple charges,
(S3) multiple defendants with a single charge, and
(S4) multiple defendants with multiple charges.
We evaluate the dataset across two Legal judgment prediction tasks, i.e., charge prediction and penalty term prediction.
We have conducted extensive experiments and found that the scenario involving multiple defendants and multiple charges (S4) poses the greatest challenges, followed by S2, S3, and S1.
The impact varies significantly depending on the model.
For example, in S4 compared to S1, InternLM2 achieves approximately 4.5\% lower F1-score and 2.8\% higher LogD, while Lawformer demonstrates around 19.7\% lower F1-score and 19.0\% higher LogD.
Paper Type: Short
Research Area: Resources and Evaluation
Research Area Keywords: Legal judgment prediction, large language models, text generation
Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models, Data resources, Data analysis
Languages Studied: Chinese
Submission Number: 5493
Loading