Latency Driven Spatially Sparse Optimization for Multi-Branch CNNs for Semantic Segmentation

Georgios Zampokas, Christos-Savvas Bouganis, Dimitrios Tzovaras

Abstract: Semantic segmentation has gained significant attention in the field of computer vision, especially in the context of autonomous driving. Achieving superior performance and precise object localization is paramount for safe and reliable autonomous vehicles. This introduces the need to process high-resolution feature maps, resulting in increased computational requirements. Recently proposed multi-branch architectures address that by maintaining parallel computationally light high-resolution representations throughout the whole network. Since individual branches focus on different image regions by design, we believe that there is significant number of redundant computations, especially in high-resolution branches. To harness that, we propose a optimization scheme for multi-branch CNNs, which introduces spatial sparsity to the network to produce more efficient distribution of calculations. The proposed approach departs from the literature by introducing the actuallatency in the optimization process, resulting in device-tailored and practically-efficient sparse architectures.