Certifying Reading Comprehension in Large Language Models

Isha Chaudhary, Vedaant V Jain, Gagandeep Singh

TL;DR: We present the first formal certification framework for LLM reading comprehension.

Abstract:

Large Language Models (LLMs) are increasingly deployed in safety-critical systems
that rely heavily on reading comprehension—extracting and reasoning over exten-
sive in-context information. However, existing evaluations of LLMs on reading comprehension are typically over limited test sets
containing only a tiny fraction of the vast
number of possible prompts. Empirical evaluations on these test sets have questionable
reliability and generalizability. We propose
a fundamentally different approach: rather
than evaluating LLMs with fixed datasets,
we introduce the first framework for certifying LLMs based on large probability distributions over realistic reading comprehension
prompts. To create these distributions, we
use knowledge graphs (KGs) as structured
representations of real-world knowledge and
define the distributions’ sample spaces with
prompts based on directed acyclic subgraphs
of the KGs. We also incorporate realistic
noise designed to mimic real-world complexity, such as distractor texts and synonyms.
Our prompt distributions have i.i.d. samplers represented as probabilistic programs.
Our framework generates novel, formal probabilistic quantitative certificates that provide
high-confidence, tight bounds on the probability that an LLM correctly answers any
prompt drawn from these distributions. We
enable formal certification for SOTA LLMs
by using an input-output example-driven approach. We apply our framework to certify
SOTA LLMs in precision medicine and general question-answering domains. Our results
uncover previously unknown vulnerabilities caused by natural prompt noise and establish the first formal performance hierarchies
among these models.

Submission Number: 1099