SimuCourt: Building Judicial Decision-Making Agents with Real-world Judgement Documents

Anonymous

Abstract: With the development of deep learning, natural language processing technology has effectively improved the efficiency of various aspects of the traditional judicial industry. However, most current efforts focus solely on individual judicial stage, overlooking cross-stage collaboration. As the autonomous agents powered by large language models are becoming increasingly smart and able to make complex decisions in real-world settings, offering new insights for judicial intelligence. In this paper, (1) we introduce SimuCourt, a judicial benchmark that encompasses 420 judgment documents, spanning the three most common types of judicial cases, and a novel task Judicial Decision Making to evaluate the judicial analysis and decision-making power of agents. To support this task, we construct a large-scale judicial knowledge base, JudicialKB, with multiple legal knowledge. (2) we propose a novel multi-agent framework, AgentsCourt. Our framework follows the real-world classic court trial process, consisting of court debate simulation, legal information retrieval and judgement refinement to simulate the decision making of judge. (3) we perform extensive experiments, the results demonstrate that, compared to the existing advanced methods, our framework outperforms the existing advanced methods in various aspects, especially in generating legal grounds, where our system achieves significant improvements of 8.6% and 9.1% F1 score in the first and second instance settings, respectively.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment, Data resources

Languages Studied: Chinese