Combining street imagery and spatial information for measuring socioeconomic status

Esra Suel, Marthe Boulleau, Majid Ezzati, Seth Flaxman

Keywords: measurement of socioeconomic status, poverty mapping, population mapping, deep learning, Gaussian processes, spatial statistics

TL;DR: We empirically compare GP and NN based methods for incorporating street view imagery and spatial information for measuring socioeconomic status indicators in London.

Abstract: Emerging sources of large-scale data, such as remote sensing, street imagery, GPS trajectories, coupled with advances in deep learning methods have the potential for significantly advancing how fast, how frequently, and how locally we can measure urban features and population characteristics to inform and evaluate policies. One such example that attracted increasing attention from the research community is utilizing street imagery from Google Street View (GSV) for various measurement tasks in this broader context. We believe incorporating spatial information with Gaussian Processes (GPs) can give us better performance when using GSVs. To test this hypothesis, we empirically investigated multiple approaches for combining spatial and street image information using neural networks and GPs for predicting income, crowding, and education levels in London, UK. Results demonstrated using GPs only with spatial information (without any inputs from images) gives us a good baseline. Complementary value of GSV images were demonstrated for the socioeconomic status measures we investigated. Further, our results showed superior performance of GP regression of residuals compared to other methods including feeding spatial information as input directly to neural networks.