SegBlocks: Towards Block-Based Adaptive Resolution Networks for Fast Segmentation

Thomas Verelst, Tinne Tuytelaars

Abstract: We propose a method to reduce the computational cost and memory consumption of existing neural networks, by exploiting spatial redundancies in images. Our method dynamically splits the image into blocks and processes low-complexity regions at a lower resolution. Our novel BlockPad module, implemented in CUDA, replaces zero-padding in order to prevent the discontinuities at patch borders of which existing methods suffer, while keeping memory consumption under control. We demonstrate SegBlocks on Cityscapes semantic segmentation, where the number of floating point operations is reduced by 30% with only 0.2% loss in accuracy (mIoU), and an inference speedup of 50% is achieved with 0.7% decrease in mIoU.