Beyond the Bar: Generative AI as a Transformative Component in Legal Document Review

Roshanak Omrani, Eugene Yang, Evan Curtin, Nathan Reff, Tara S. Emory, Lenora Gray, Jeremy Pickens, Cristin K. Traylor, Sean Underwood, Aron J. Ahmadia, David D. Lewis

Abstract: Review for responsiveness is a recall-oriented document classification task central to civil litigation. In large legal matters, it may involve the coding of millions of documents by teams of dozens to hundreds of contract attorneys. We describe a prototype document review system based on a large language model (LLM) for replacing the first level of attorney review. Our system accepts the same guidance—a written review protocol—that would be provided to a human review team. We tested our prototype in the context of a live legal matter, evaluating both human review and our LLM-based system against a gold standard coded by expert senior attorneys. Our prototype achieved an estimated 96% recall and 60% precision without matter-specific tuning, and has numerous avenues for further improvement.