EditMark: Training-free and Harmless Watermark for Large Language Models

Shuai Li, Kejiang Chen, Jun Jiang, Jie Zhang, Kai Zeng, Tianze Chang, Weiming Zhang, Nenghai Yu

Keywords: Watermarking, Model Edit, Robust

Abstract:

Large Language Models (LLMs) have demonstrated remarkable capabilities, but their training requires extensive data and computational resources, rendering them valuable digital assets. Therefore, it is essential to watermark LLMs to protect their copyright and trace unauthorized use or resale. Existing methods for watermarking LLMs are mainly based on backdoors or knowledge injection, which require burdensome training or degrade the generation quality. To address these issues, we propose EditMark, a training-free and harmless watermarking method for LLMs based on model editing.
We observe LLM has diversity and can generate multiple logical and semantic correct answers to some open-ended questions. Therefore, we can use a watermark to generate a harmless mapping to control the LLM's answer to an open-ended question.
Inspired by this insight, EditMark involves generating a harmless mapping based on the watermark, selecting a secret key to generate watermarked inputs, and editing the outputs of LLM to align with the harmless mapping.
Extensive experiments show that EditMark can embed 8-bit watermarks into LLMs within 2 minutes, with a watermark extraction success rate close to 100%. External experiments further demonstrate that EditMark has fidelity and is robust to model fine-tuning and editing attacks.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6432