Model Extraction Attacks on DistilBERT

Amro Salman, Ayman Saeed, Khalid Elmadani

Keywords: Model Extraction, DistilBERT

Abstract:

This paper investigates model extraction attacks, where an adversary can train a
substitute model by collecting data through query access to a victim model and
steal its functionality. We use DistilBERT as the victim model due to its smaller
size and faster processing speed. The results demonstrate the effectiveness of
the model extraction attack and show that there is a relation between its success
and the similarity between the training data of the victim model and the attacker’s
queries. The study provides important insights into the security of machine learning
models.

Submission Category: Machine learning algorithms

Submission Number: 78