Less Is Better: Fooling Scene Text Recognition with Minimal Perturbations

Yikun Xu, Pengwen Dai, Xiaochun Cao

Abstract: Scene text recognition (STR) has made tremendous progress in the era of deep learning. However, the attack of the sequential STR does not attract sufficient scholarly attention. The very few existing researches to fool STR belong to white-box attacks and thus would have limitations in practical applications. In this paper, we propose a novel black-box attack on STR models, only using the probability distribution of the model output. Instead of disturbing most pixels like existing STR attack methods, our proposed approach only disturbs very few pixels and utilizes own characteristics of recurrent neural networks (RNNs) to propagate perturbations. Experiments validate the effectiveness and superiority of our attack approach.