Neural Networks for irregularly observed continuous-time Stochastic Processes

Francois W. Belletti, Alexander Ku, Joseph E. Gonzalez

Abstract: Designing neural networks for continuous-time stochastic processes is challenging, especially when observations are made irregularly. In this article, we analyze neural networks from a frame theoretic perspective to identify the sufficient conditions that enable smoothly recoverable representations of signals in L^2(R). Moreover, we show that, under certain assumptions, these properties hold even when signals are irregularly observed. As we converge to the family of (convolutional) neural networks that satisfy these conditions, we show that we can optimize our convolution filters while constraining them so that they effectively compute a Discrete Wavelet Transform. Such a neural network can efficiently divide the time-axis of a signal into orthogonal sub-spaces of different temporal scale and localization. We evaluate the resulting neural network on an assortment of synthetic and real-world tasks: parsimonious auto-encoding, video classification, and financial forecasting.

TL;DR: Neural architectures providing representations of irregularly observed signals that provably enable signal reconstruction.

Keywords: Deep Learning, Stochastic Processes, Time Series Analysis

Data: [YouTube-8M](https://paperswithcode.com/dataset/youtube-8m)