Efficient real-time semantic segmentation: accelerating accuracy with fast non-local attention

Tianye Lan, Furong Dou, Ziliang Feng, Chengfang Zhang

Abstract: As an essential aspect of semantic segmentation, real-time semantic segmentation poses significant challenge in achieving trade-off between segmentation accuracy and inference speed. Standard non-local block can effectively capture the long-range dependencies that are critical to semantic segmentation, while its huge computational cost is unacceptable for real-time semantic segmentation. To confront this issue, we propose fast non-local attention network (FNANet) with encoder–decoder structure for real-time semantic segmentation. FNANet relies on the utilization of fast non-local attention module and fast non-local attention fusion module. These modules serve the dual purpose of reducing computational demands and capturing essential contextual information, thereby achieving an equilibrium between enhanced segmentation accuracy and minimized computational overhead. Furthermore, improved non-local attention is incorporated to augment feature representation, consequently facilitating precise class label prediction. Experimental results demonstrate that FNANet outperforms state-of-the-art methods in terms of segmentation accuracy and speed on Cityscapes and CamVid.