Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction

Fuyang Zhang, Nelson Nauata, Yasutaka Furukawa

Abstract:

This paper proposes a novel message passing neural
(MPN) architecture Conv-MPN, which reconstructs an outdoor building as a planar graph from a single RGB image.
Conv-MPN is specifically designed for cases where nodes
of a graph have explicit spatial embedding. In our problem, nodes correspond to building edges in an image. ConvMPN is different from MPN in that 1) the feature associated
with a node is represented as a feature volume instead of a
1D vector; and 2) convolutions encode messages instead of
fully connected layers. Conv-MPN learns to select a true
subset of nodes (i.e., building edges) to reconstruct a building planar graph. Our qualitative and quantitative evaluations over 2,000 buildings show that Conv-MPN makes
significant improvements over the existing fully neural solutions. We believe that the paper has a potential to open
a new line of graph neural network research for structured
geometry reconstruction.