Scene text recognition via dual character counting-aware visual and semantic modeling network

Ke Xiao, Anna Zhu, Brian Kenji Iwana, Cheng-Lin Liu

Abstract: In this work, we study character counting in STR from a new viewpoint, giving a principled framework showing that the counting information is involved in both visual decoding and semantic decoding. Based on the principled framework, we propose a novel scene text recognizer with a dual character counting-aware visual and semantic modeling network, where the counting information is fused in both vision and language branches. Experimental results demonstrate the effectiveness of our model.