HSNet: A Novel Edge-Preserving Hierarchical Separable Network for Video Shadow Detection

Hemraj Singh, Mridula Verma, Ramalingaswamy Cheruku

Abstract:

Video Shadow Detection (VSD) is an emerging research direction that holds significant importance in surveillance video analysis for multiple industrial applications. Recently, several research efforts in the field of VSD have focused on improving the performance and accuracy of shadow detection by utilizing state-of-the-art deeplearning models, overlooking the challenges associated with practical deployment on resource-constrained devices. To maintain this trade-off between accuracy and computational complexity, we propose a novel edge-preserving lightweight Hierarchical Separable Network (HSNet) for VSD tasks, which hierarchically extracts the attentionbased multi-scale geometric spatiotemporal shadow features from videos to improve shadow detection performance while keeping the number of network parameters and floating point operations low. As far as we know, this is the first work that extracts the attention-based multi-scale geometric spatial and temporal features hierarchically. Additionally, a Geometric Attention Information Module (GAIM) is designed, which
extracts geometric spatial and temporal resolution information from video frames and preserves the edge information. Next, a novel Edge-enhanced Detection Network (EDNet) is proposed to extract geometric spatial and temporal features and enhance edge information. To enhance the diversity of the existing datasets with visually complex shadow scene variations, we collected new annotated examples. Lastly, Shadow Region Intensity (SRI) loss is proposed to minimize the training loss and differentiate the geometric variation of the background and foreground of the objects. Extensive experimental results demonstrate that HSNet outperformed existing state-of-the-art models with 2.82% of increased accuracy while achieving 87.9 and 81.4% reduction in the number of parameters and FLOPS, respectively, on VSD. Our code and data samples and the corresponding annotations are available at https://github.com/shemraj/HSNet.