Single-Loss Multi-task Learning For Improving Semantic Segmentation Using Super-Resolution

Andreas Aakerberg, Anders Skaarup Johansen, Kamal Nasrollahi, Thomas B. Moeslund

Abstract: We propose a novel means to improve the accuracy of semantic segmentation based on multi-task learning. More specifically, in our Multi-Task Semantic Segmentation and Super-Resolution (MT-SSSR) framework, we jointly train a super-resolution and semantic segmentation model in an end-to-end manner using the same task loss for both models. This allows us to optimize the super-resolution model towards producing images that are optimal for the segmentation task, rather than ones that are of high-fidelity. Simultaneously we adapt the segmentation model to better utilize the improved images and thereby improve the segmentation accuracy. We evaluate our approach on multiple public benchmark datasets, and our extensive experimental results show that our novel MT-SSSR framework outperforms other state-of-the-art approaches.

External IDs:dblp:conf/caip/AakerbergJNM21