Legal Retrieval for Public Defenders

TMLR Paper7389 Authors

Abstract:

AI tools are increasingly suggested as solutions to assist public agencies with heavy workloads. In public defense---where a constitutional right to counsel meets the complexities of law, overwhelming caseloads, and constrained resources---practitioners face especially taxing conditions. Yet, there is little evidence of how AI could meaningfully support defenders' day-to-day work. In partnership with the anonymized Office of the Public Defender, we develop the anonymized BriefBank, a retrieval tool which surfaces relevant appellate briefs to streamline legal research and writing.
We show that existing retrieval benchmarks fail to transfer to real public defense research, however adding domain knowledge improves retrieval quality. This includes query expansion with legal reasoning, domain-specific data and curated synthetic examples. To facilitate further research, we release a new, realistic retrieval dataset, manually annotated by real public defenders, and provide a taxonomy of these realistic defender search queries.
Together, our work improves on the status quo of realistic retrieval benchmarking and provides a starting point for leveraging AI in a real-world public interest setting.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Huaxiu_Yao1

Submission Number: 7389