ArtistAuditor: Auditing Artist Style Pirate in Text-to-image Generation Models
Track: Security and privacy
Keywords: trustworthy machine learning, dataset copyright auditing, text-to-image model
TL;DR: To address the concerns about artworks’ copyright infringement, we propose a novel method for data-use auditing in the text-to-image generation model.
Abstract: The text-to-image models based on diffusion processes, such as DALL-E, Stable Diffusion, and Midjourney, are capable of transforming texts into detailed images and have widespread applications in art and design. As such, amateur users can easily imitate professional-level paintings by collecting an artist’s work and fine-tuning the model, leading to concerns about artworks’ copyright infringement. To tackle these issues, previous studies either add visually imperceptible perturbation to the artwork to change its underlying styles (perturbation-based methods) or embed posttraining detectable watermarks in the artwork (watermark-based methods). However, when the artwork or the model has been published online, i.e., modification to the original artwork or model retraining is not feasible, these strategies might not be viable.
To this end, we propose a novel method for data-use auditing in the text-to-image generation model. The general idea of ArtistAuditor is to identify if a suspicious model has been fine-tuned using specific artists’ artworks by analyzing style-related features. Concretely, ArtistAuditor employs a style extractor to obtain the multi-granularity style representations and treats artworks as samplings of an artist’s style. Then, ArtistAuditor queries a trained discriminator to gain the auditing decisions. The experimental results on six combinations of models and datasets show that ArtistAuditor can achieve high AUC values (> 0.937). By studying ArtistAuditor’s transferability and core modules, we provide valuable insights into the practical implementation. Finally, we demonstrate the effectiveness of ArtistAuditor in real-world cases by an online platform Scenario.1 ArtistAuditor is open-sourced at https://anonymous.4open.science/r/ArtistAuditor.
Submission Number: 1384
Loading