Blind Judgement: Agent-Based Supreme Court Modelling With GPT

Sil Hamilton

Keywords: multi-agent system, natural language generation, social simulation, supreme court

TL;DR: We show how GPT-2 accurately predicts Supreme Court decisions despite hallucinating precedence.

Abstract: We present a novel Transformer-based multi-agent system for simulating the judicial rulings of the 2010-2016 Supreme Court of the United States. We train nine separate models with the respective authored opinions of each supreme justice active ca. 2015 and test the resulting system on 96 real-world cases. We find our system predicts the decisions of the real-world Supreme Court with better-than-random accuracy. We further find a correlation between model accuracy with respect to individual justices and their alignment between legal conservatism & liberalism. Our methods and results hold significance for researchers interested in using language models to simulate politically-charged discourse between multiple agents.

Submission Type: archival

Presentation Type: online

Presenter: Sil Hamilton