Entropy Proxy for LLM Memorization Score

Yizhan Huang, Zhe YANG, Meifang Chen, HUANG Nianchen, Jianping Zhang, Michael R. Lyu

Keywords: Memorization, Entropy, LLMs

Abstract: Large Language Models (LLMs) are known to memorize portions of their training data, sometimes reproducing content verbatim when prompted appropriately. Existing memorization research rarely explores how training data influences memorization and often limits the experimental setup to a binarized memorization vs non-memorization catagory. In this work, we investigate a fundamental yet under-explored question in the domain of memorization: How to quantitatively characterize memorization difficulty using intrinsic properties of training data in LLMs? Inspired by early studies using compression algorithms to filter out simple memorization cases, we explore the link between training data compressibility and memorization. Through experiments on a wide range of open models without various setups, we present the Entropy–Memorization Law. It suggests that at the set-level, data entropy (estimator) is linearly correlated with memorization score. We also further investigate EM Law with several dimensions: visualizing vocabulary size as an implicit factor, and applying the law to data with disparate semantics.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 2818