Toward Breaking Watermarks in Distortion-free Large Language Models

Shayleen Reynolds, Saheed Obitayo, Niccolo Dalmasso, Dung Daniel Ngo, Vamsi K. Potluru, Manuela Veloso

Keywords: Watermarking, Large Language Models, Linear Programming, Optimization

Abstract: In recent years, LLM watermarking has emerged as an attractive safeguard against AI-generated content, with promising applications in many real-world domains. However, there are growing concerns that the current LLM watermarking schemes are vulnerable to expert adversaries wishing to reverse-engineer the watermarking mechanisms. Prior work in ‘breaking’ or ‘stealing’ LLM watermarks mainly focuses on the distribution-modifying algorithm of Kirchenbauer et al. (2023), which perturbs the logit vector before sampling. In this work, we focus on reverse-engineering the other prominent LLM watermarking scheme, distortion-free watermarking (Kuditipudi et al. 2024), which preserves the underlying token distribution by using a hidden watermarking key sequence. We demonstrate that, even under a more sophisticated watermarking scheme, it is possible to ‘compromise’ the LLM and carry out a ‘spoofing’ attack. Specifically, we propose a mixed integer linear programming framework that accurately estimates the secret key used for watermarking using only a few samples of the watermarked dataset. Our initial findings challenge the current theoretical claims on the robustness and usability of existing LLM watermarking techniques.

Submission Number: 29