Temperature-Based Watermarking and Detection for Large Language Models

Weitong Chen, Zhenxin Zhang, Huali Ren, Pei-Gen Ye, Zhengdao Li, Shanshan Huang

Abstract: With the wide application of Large Language Models (LLMs), protecting the copyright of generated content and preventing its misuse becomes important. This paper proposes a temperature-based watermark embedding algorithm that embeds watermarks in text using the Softmax function and polynomial sampling techniques. Meanwhile, this paper also discusses a watermark detection technique based on statistical testing, which can effectively identify and verify watermarks embedded in text. By applying these techniques to different LLMs and computing environments, including OPT series, Llama series, BLOOM series and GPT-2, this paper analyses the scenarios, evaluates the key parameters in the algorithms and proposes solutions to ensure the integration of watermarks without compromising on the performance of the model or the naturalness of the generated text.