EKENet: Efficient knowledge enhanced network for real-time scene parsing

Ao Luo, Fan Yang, Xin Li, Rui Huang, Hong Cheng

Abstract: Highlights • We propose an Efficient Dual Abstraction block, which is able to extract rich features with low computation complexity. • We introduce a novel light-weight Encoding-Enhancing module to enhance the representation of high-resolution feature map. • Our fully-equipped model (EKENet), achieves the new state-of-the-art performance in terms of speed and accuracy tradeoff. Abstract Scene parsing is essential for many high-level AI applications, such as intelligent vehicles and traffic surveillance. In this work, we propose a highly efficient and powerful deep convolutional neural network, namely Efficient Knowledge Enhanced Network (EKENet), for parsing scenes in real-time. Unlike most existing approaches that compromise efficiency for the sake of high accuracy, EKENet achieves an ideal trade-off between the two. Our EKENet is built upon a novel building block, namely Efficient Dual Abstraction (EDA) block, which employs an efficiently parallel convolution structure for extracting spatial features and modeling cross-channel correlations in a dual fashion. Additionally, a novel light-weight Encoding-Enhancing (EE) module is designed to enhance our EKENet, which can efficiently encode high-level knowledge extracted from top layers to guide the learning of low-level features from bottom layers. Extensive experiments on challenging benchmarks, Cityscapes and CamVid datasets, demonstrate that EKENet achieves the new state-of-the-art performance in terms of speed and accuracy tradeoff.