How to Fool Systems and Humans in Visually Grounded Interaction: A Case Study on Adversarial Attacks on Visual Dialog

Anonymous

Abstract: Adversarial attacks change predictions of deep neural network models, while aiming to remain unnoticed by the user.This is a challenge for textual attacks, which target discrete text. In this study, we investigate the robustness of visually grounded dialog models towards textual attacks to understand how different input components can mitigate the attack. Our results show that dialog history is important for model robustness: models encoding history are more robust, and when launching an attack on history, model prediction becomes more uncertain. This is in contrast to prior work which finds that dialog history is negligible for model performance. We also evaluate how to generate adversarial examples which successfully attack the model but remain undetected by the user. We find that the textual, as well as the visual context is important to generate attacks which appear semantically coherent to humans.

Paper Type: long