Aerial imagery for roof segmentation A large-scale dataset towards automatic mapping of buildings

Lei Wang, Qi Chen, Steven L. Waslander

Abstract:

The identification and annotation of buildings has long been a tedious and expensive part of high-precision
vector map production. The deep learning techniques such as fully convolution network (FCN) have largely
promoted the accuracy of automatic building segmentation from remote sensing images. However, compared
with the deep-learning-based building segmentation methods that greatly benefit from data-driven feature
learning, the building boundary vector representation generation techniques mainly rely on handcrafted
features and high human intervention. These techniques continue to employ manual design and ignore the
opportunity of using the rich feature information that can be learned from training data to directly generate
vectorized boundary descriptions. Aiming to address this problem, we introduce PolygonCNN, a learnable endto-end vector shape modeling framework for generating building outlines from aerial images. The framework
first performs an FCN-like segmentation to extract initial building contours. Then, by encoding the vertices of
the building polygons along with the pooled image features extracted from segmentation step, a modified
PointNet is proposed to learn shape priors and predict a polygon vertex deformation to generate refined
building vector results. Additionally, we propose 1) a simplify-and-densify sampling strategy to generate
homogeneously sampled polygon with well-kept geometric signals for shape prior learning; and 2) a novel
loss function for estimating shape similarity between building polygons with vastly different vertex numbers.
The experiments on over 10,000 building samples verify that PolygonCNN can generate building vectors with
higher vertex-based F1-score than the state-of-the-art method, and simultaneously well maintains the building
segmentation accuracy achieved by the FCN-like model.