Towards Scene Text Recognition in Rainy Weather Conditions

Anandita Jamwal, Lalithya Koneti, Manikandan Ravikiran, Dinesh Singh, Rohit Saluja

Abstract: Scene Text Recognition (STR) is an essential component in autonomous navigation for a diverse set of applications, including reading signboards, traffic signs and license plates. However, STR in rainy weather conditions remains unexplored. The challenges of reading text covered with rain streaks and accumulation highlight the need for robust STR systems. We adapt existing rain generation method to add rain to the 1000 text-heavy frames from the RoadText-1K dataset. RoadText-1K is an existing dataset that is created from the Berkeley Deep Drive (BDD) dataset. We observe challenges like completely illegible text and non-real rains while using the rain generation methods. We first provide a simple recipe to build somewhat legible but challenging STR scenes in real rainy weather. The recipe includes upsampling the noisy rain mask obtained from the rain generation method, and adding it to the high-resolution clean image. Although our method is simple, the generated data helps improve STR results in real rainy weather. We benchmark the performance of existing STR models on our data. Further fine-tuning the best-performing model on generated rainy data reduces the word error rates by 27.88% on generated test set, and 22.36% on images with real rain. It is important to note that these results are better than directly fine-tuning on data with real rain, since the labelling was done before adding rain, and the scene text is challenging but readable.

External IDs:dblp:conf/icdar/JamwalKRSS25