Target Model Agnostic Adversarial Attacks with Query Budgets on Language Understanding Models

Jatin Chauhan, Karan Bhukar, Manohar Kaul

Abstract:

Despite significant improvements in natural
language understanding models with the advent of models like BERT and XLNet, these
neural-network based classifiers are vulnerable to blackbox adversarial attacks, where the
attacker is only allowed to query the target
model outputs. We add two more realistic restrictions on the attack methods, namely limiting the number of queries allowed (query
budget) and crafting attacks that easily transfer across different pre-trained models (transferability), which render previous attack models impractical and ineffective. Here, we propose a target model agnostic adversarial attack
method with a high degree of attack transferability across the attacked models. Our empirical studies show that in comparison to baseline
methods, our method generates highly transferable adversarial sentences under the restriction
of limited query budgets.