Stereollax Net: Stereo Parallax-Based Deep Learning Network for Building Height Estimation

Sana Jabbar, Murtaza Taj

Abstract: Accurate estimation of building heights is crucial for effective urban planning and resource management as it provides essential geometric information about the urban landscape. Many end-to-end deep learning-based networks have been proposed for image-to-height mapping using high-resolution nonoptical and optical remote sensing imagery. In this study, we develop a novel deep-learning architecture that incorporates a stereo parallax-based mathematical formulation for building height estimation. We estimate stereo formulation parameters include differential parallax ( $ \Delta \mathbf {P}$ ) image, average photo-base ( $b$ ), and satellite height ( $h_{s}$ ). The final height map is computed by utilizing these parameters in the stereo parallax equation, thus combining closed-form solutions within the learning paradigm. Moreover, to improve the estimation of $ \Delta \mathbf {P}$ , we also introduce a multiscale differential shortcut connection (MSDSC) module. The MSDSC module integrates high-frequency components into lower resolution baseline decoder features while converting them into high-resolution decoder features. To establish the efficacy of our proposed stereo parallax-based deep learning network (Stereollax Net), we train and evaluate our method on densely populated cities of China (42-Cities dataset) and on the IEEE Data Fusion Contest 2018 (DFC2018) dataset. Our proposed Stereollax Net is trained only with RGB imagery and compared with the state-of-the-art (SOTA) methods that utilize both panchromatic and multispectral (RGB and near-infrared) satellite imagery. The qualitative and quantitative results demonstrate that our Stereollax Net surpasses existing SOTA algorithms, achieving superior performance with fewer data and training parameters by a considerable margin. The code will be made publicly available via the GitHub repository.