Probabilistic Reasoning with LLMs for Privacy Risk Estimation

Jonathan Zheng, Alan Ritter, Sauvik Das, Wei Xu

Keywords: LLMs, probabilistic reasoning, chain of thought, prompting, NLP, privacy risk, personal disclosures

TL;DR: We introduce a new probabilistic method and task for quantifying the privacy risk of a document with personal attributes using large language models.

Abstract: Probabilistic reasoning is a key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision-making. In this paper, we introduce a new numerical reasoning task under uncertainty for large language models, focusing on estimating the privacy risk of user-generated documents containing privacy-sensitive information. We propose BRANCH, a new LLM methodology that estimates the $k$-privacy value of a text—the size of the population matching the given information. BRANCH factorizes a joint probability distribution of personal information as random variables. The probability of each factor in a population is estimated separately using a Bayesian network and combined to compute the final $k$-value. Our experiments show that this method successfully estimates the $k$-value 73% of the time, a 13% increase compared to o3-mini with chain-of-thought reasoning. We also find that LLM uncertainty is a good indicator for accuracy, as high variance predictions are 37.47% less accurate on average.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 25145