Learning General Representations for Semantic Segmentation and Height Estimation from Remote Sensing Images

Wufan Zhao, Claudio Persello, Hu Ding, Alfred Stein

Abstract: Deep-learning methods have been applied to remote-sensing imagery to achieve height estimation and semantic segmentation. Recent research has demonstrated that multi-task learning methods can act as a welcome addition to task-specific features by improving prediction accuracy across a range of tasks. How to effectively learn representations that achieve good performance for multiple tasks still remains challenging. We propose to adopt a unified network jointly learning multiple vision tasks’ general representations by aligning them via small task-specific adapters. The experimental results on the Vaihingen dataset demonstrate that general representation learning can improve the performance of state-of-the-art methods in height estimation and semantic segmentation tasks.