DevianceNet: Learning to Predict Deviance from A Large-scale Geo-tagged Dataset

Jin-Hwi Park, Young-Jae Park, Junoh Lee, Hae-Gon Jeon

Abstract:

Understanding how a city’s physical appearance and environmental surroundings impact society traits, such as safety, is
an essential issue in social artifcial intelligence. To demonstrate the relationship, most existing studies utilize subjective
human perceptual attributes, categorization only for a few violent crimes, and images taken from still shot images. These
lead to diffculty in identifying location-specifc characteristics for urban safety. In this work, to address this problem, we
propose a large-scale dataset and a novel method by adopting
a concept of “Deviance” which explains behaviors violating
social norms, both formally (e.g. crime) and informally (e.g.
civil complaints). We frst collect a geo-tagged dataset consisting of incident report data for seven metropolitan cities,
with corresponding sequential images around incident sites
obtained from Google street view. We also design a convolutional neural network that learns spatio-temporal visual attributes of deviant streets. Experimental results show that our
framework can reliably recognize real-world deviance in various cities. Furthermore, we analyze which visual attribute is
important for deviance identifcation and severity estimation.
We have released our dataset and source codes at our project
page: https://deviance-project.github.io/DevianceNet/