MASK-CNN-Transformer for real-time multi-label weather recognition

Shengchao Chen, Ting Shu, Huan Zhao, Yuan Yan Tang

Abstract: Highlights•For enhanced real-world weather recognition, we introduce MASK-CNN-Transformer. This model fuses pre-trained CNN and Transformer to capture intricate feature-context connections in outdoor images.•To enhance model’s generalization and recognition, we propose MASK training strategy. It randomly selects image regions and labels, fostering global-local feature connections and interactive weather learning.•Validated MASK-CT’s effectiveness via real-world dynamic scenarios with a real-time weather dataset.•MASK-CT achieves SOTA weather recognition performance on real-world data, with dynamic real-time recognition up to 101.3 FPS.