Model Extraction Attacks on DistilBERT

Amro Salman, Ayman Saeed, Khalid Elmadani, Sharief Babiker

Keywords: Model Extraction, DistilBERT

Abstract: This paper investigates model extraction attacks, where an adversary can train a substitute model by collecting data through query access to a victim model and stealing its functionality. We use DistilBERT as the victim model due to its smaller size and faster processing speed. The results demonstrate the effectiveness of the model extraction attack and show that fine-tuning more powerful language models can improve accuracy. The study provides important insights into the security of machine learning models.