Go to ACL ARR 2025 February homepageUnlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland
Abstract: Legal research is a time-consuming task that most lawyers face on a daily basis. A large part of legal research entails looking up relevant caselaw and bringing it in relation to the case at hand. Lawyers heavily rely on summaries (also called headnotes) to find the right cases quickly. However, not all decisions are annotated with headnotes and writing them is time-consuming. Automated headnote creation has the potential to make hundreds of thousands of decisions more accessible for legal research in Switzerland alone. To address this, we introduce the Swiss Landmark Decisions Summarization (SLDS) dataset, a cross-lingual resource with 20K landmark rulings from the Swiss Federal Supreme Court, each with headnotes in German, French, and Italian. We fine-tune models from the Qwen2.5, Llama 3.2, and Phi-3.5 families and compare them to larger proprietary models, including GPT-4o and Claude 3.5 Sonnet, and DeepSeek R1. While fine-tuned models achieve high lexical similarity, proprietary models excel in legal accuracy and coherence, as shown by an LLM-as-a-Judge evaluation.Our evaluation reveals that while fine-tuned models achieve strong lexical similarity, proprietary models generate more legally accurate and structured headnotes. Surprisingly, reasoning models do not significantly outperform general-purpose LLMs, indicating that structured factual accuracy is more crucial than deep logical reasoning in judicial summarization. To advance research in cross-lingual legal summarization, we release SLDS under a CC BY 4.0 license.
Paper Type: Long
Research Area: Summarization
Research Area Keywords: Generation, Multilingualism and Cross-Lingual NLP, Resources and Evaluation, Summarization
Languages Studied: German, French, Italian
Submission Number: 8458
Loading