Erasing the Invisible: A Stress-Test Challenge for Image Watermarks

Mucong Ding, Tahseen Rabbani, Bang An, Souradip Chakraborty, Chenghao Deng, Mehrdad Saberi, Yuxin Wen, Xuandong Zhao, Mo Zhou, Anirudh Satheesh, Mary-Anne Hartley, Lei Li, Yu-Xiang Wang, Vishal M. Patel, Soheil Feizi, Tom Goldstein, Furong Huang

Keywords: Responsible AI, Generative AI, Image Watermarks, AI-generated Content Detection, Red Teaming

TL;DR: "Erasing the Invisible" invites innovators to breach and bolster image watermarking methods through a dynamic, dual-track competition of watermark stress-testing, utilizing a 10k image set for rolling evaluations and a 50k image grand test.

Abstract: "Erasing the Invisible" is a pioneering competition designed to rigorously stress-test image watermarks, aiming to enhance their robustness significantly. Its standout feature is the introduction of dual tracks for black-box and beige-box attacks, providing a nuanced approach to validate the reliability and robustness of watermarks under varied conditions of visibility and knowledge. The competition spans from July 18 to October 31, inviting individuals and teams to register and participate in a dynamic challenge. Throughout the competition, employing a dataset of 10k images accessed through the Hugging Face API, competitors will receive updated evaluation results on a rolling basis and submit their refined techniques for the final evaluation, which will be conducted on an extensive dataset of 50k images. The evaluation process of this competition not only emphasizes the effectiveness of watermark removal but also highlights the critical importance of maintaining image quality, with results reflected on a continuously updated leaderboard. "Erasing the Invisible" promises to elevate watermarking technology to new heights of resilience, setting a precedent for future research and application in digital content security and safeguarding against unauthorized use and misinformation in the digital age.

Competition Timeline:

- Organizers prepare the event: April 17 - July 17
- Competition opens: July 18
- Development phase: July 18 - October 24
- Registration closes: October 24
- Final submission phase: October 24 - October 31
- Competition closes: October 31
- Organizers evaluate final submissions: November 1 - November 8
- Winning team announcement: November 9
- Organizers prepare the competition workshop: November 9 - December 9

Website: https://erasinginvisible.github.io/

Primary Contact Email: [e-mail]

Participant Contact Email: [e-mail]

Workshop Format: Hybrid (Vancouver + some online speakers)

Preferred Timezone: Eastern Time Zone

Submission Number: 48