Watermarking Language Models with Error Correcting Codes

Patrick Chao, Yan Sun, Edgar Dobriban, Hamed Hassani

Track: long paper (up to 9 pages)

Keywords: Large Language Model; Watermarking; Error Correcting Code

TL;DR: We propose a watermarking framework that embeds statistical signals in the output through an error correcting code.

Abstract: Recent progress in large language models enables the creation of realistic machine-generated content. Watermarking is a promising approach to distinguish machine-generated text from human text, embedding statistical signals in the output that are ideally undetectable to humans. We propose a watermarking framework that encodes such signals through an error correcting code. Our method, termed robust binary code (RBC) watermark, introduces no noticeable degradation in quality. We evaluate our watermark on base and instruction fine-tuned models and find our watermark is robust to edits, deletions, and translations. We provide an information-theoretic perspective on watermarking, a powerful statistical test for detection and for generating $p$-values, and theoretical guarantees. Our empirical findings suggest our watermark is fast, powerful, and robust, comparing favorably to the state-of-the-art.

Presenter: ~Yan_Sun6

Format: No, the presenting author is unable to, or unlikely to be able to, attend in person.

Funding: No, the presenting author of this submission does *not* fall under ICLR’s funding aims, or has sufficient alternate funding.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 13