Corer: Concept Residue Erasing in Text-to-Image Diffusion Models

Yufan Liu, Jinyang An, Huashan Chen, Wanqian Zhang, Ming Li, Dayan Wu, Jingzi Gu, Zheng Lin, Weiping Wang

Abstract: The remarkable development of text-to-image generation models has raised notable security concerns, such as the infringement of portrait rights and the generation of inappropriate content. Concept erasure has been proposed to remove the model’s knowledge about protected or inappropriate concepts. Although many methods have tried to balance the efficacy (erasing target concepts) and specificity (retaining irrelevant concepts), they can still generate abundant erasure concepts under the steering of semantically related inputs. In this work, we propose Corer to address this "concept residue" issue. Specifically, we first introduce the mechanism of neighbor-concept mining to dig out the associated concepts and expand the erasing range. Furthermore, to mitigate the negative impact on the generation of irrelevant concepts caused by the expansion of erasure scope, Corer preserves the specificity through the beyond-concept regularization. We also employ the closed-form solution to optimize weights of U-Net, as well as the prediction noise alignment with the LoRA module. Extensive experiments on multiple benchmarks demonstrate that Corer outperforms previous concept-erasing methods in terms of superior erasing efficacy, specificity, and generality.

External IDs:dblp:conf/icmcs/LiuACZLWGLW25