Racial skew in fine-tuned legal AI language models

Anamika Kumari

Abstract: Rapid growth in the application of large language models to an immense variety of use-case scenarios has occurred alongside increasing concern that such models learn, encode, and perpetuate harmful social biases. This concern merits particular scrutiny in legal contexts, which involve search, research, and reasoning tasks that stand to benefit greatly from LLMs, but also demand adherence to rigorous standards of fairness and equality before the law. In this paper, we operationalize legal equality in the U.S. legal context as a "blind" language model that, when prompted to select a race to fill in a blank, assigns equal probability to all choices. We then fine-tune a pretrained GPT-2 model on multiple subsets of American case law texts, accounting for time and the political lean of each court’s host state. We identify and measure the degree to which these models are unfair, deviating from legal equality by learning to associate different races with different legal contexts.