Data-Free Encoder Stealing Attack in Self-supervised Learning

Chuan Zhang, Xuhao Ren, Haotian Liang, Qing Fan, Xiangyun Tang, Chunhai Li, Liehuang Zhu, Yajie Wang

Abstract: Self-supervised learning technology has rapidly developed in making full use of unlabeled images, using large amounts of unlabeled data to pre-train encoders, which has led to the rise of Encoder as a Service (EaaS). The demands of large amounts of data and computing resources put pre-trained encoders at risk of stealing attacks, which is an easy way to acquire encoder functionality cheaply. Conventional attacks against encoders assume the adversary can possess a surrogate dataset with a distribution similar to that of the proprietary training data employed to train the target encoder. In practical terms, this assumption is impractical, as obtaining such a surrogate dataset is expensive and difficult. In this paper, we propose a novel data-free encoder stealing attack called DaES. Specifically, we introduce a generator training scheme to craft synthetic inputs used for minimizing the distance between the embeddings of the target encoder and surrogate encoder. This approach enables the surrogate encoder to mimic the behavior of the target encoder. Furthermore, we employ gradient estimation methods to overcome the challenge posed by limited black-box access to the target encoder, thereby improving the attack’s efficiency. Our experiments conducted across various encoders and datasets illustrate that our attack enhances state-of-the-art accuracy by up to 6.20%.