Encoding Spatio-temporal Locations with Orthogonal Function Representations

David Mickisch, Konstantin Klemmer, Mélisande Teng, Esther Rolf, Marc Rußwurm, David Rolnick

Keywords: climate and weather, surrogate modelling, geographic time series, location encodings, deep learning regularization

TL;DR: We investigate different ways to encode time for spatio-temporal interpolation tasks as well as a novel regularization for orthogonal function representations.

Abstract:

Complex spatio-temporal dependencies govern many real-world processes -- from climate dynamics to disease spread.
Modeling these processes continuously using purpose-built neural network architectures, so-called location encoders, presents an emerging paradigm in analyzing and interpolating geographic data. In this work, we expand existing spatial location encoders and introduce a new time-informed architecture: the space-time encoder.
Our method takes in geographic (latitude, longitude) and temporal information simultaneously and learns smooth, continuous functions in space and time. The inputs are first transformed using positional encoding functions and then fed into neural networks that allow the learning of complex functions.
We consider, via detailed experimental analysis, (1) how to integrate space and time encodings, (2) the effect of different choices of encoding functions for the time component and (3) frameworks for encouraging orthogonality of feature representations to improve representational power. We highlight the effectiveness and flexibility of the space-time encoder on a range of tasks representing different spatio-temporal dynamics, from climate prediction to animal species classification.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 18609