Blind Judgement: Agent-Based Supreme Court Modelling With GPTDownload PDF

21 Nov 2022 (modified: 05 May 2023)creativeAIReaders: Everyone
Keywords: multi-agent system, natural language generation, social simulation, supreme court
TL;DR: We show how GPT-2 accurately predicts Supreme Court decisions despite hallucinating precedence.
Abstract: We present a novel Transformer-based multi-agent system for simulating the judicial rulings of the 2010-2016 Supreme Court of the United States. We train nine separate models with the respective authored opinions of each supreme justice active ca. 2015 and test the resulting system on 96 real-world cases. We find our system predicts the decisions of the real-world Supreme Court with better-than-random accuracy. We further find a correlation between model accuracy with respect to individual justices and their alignment between legal conservatism & liberalism. Our methods and results hold significance for researchers interested in using language models to simulate politically-charged discourse between multiple agents.
Submission Type: archival
Presentation Type: online
Presenter: Sil Hamilton
0 Replies