Deep Direct Discriminative Decoders for High-dimensional Time-series Data Analysis

Mohammad Reza Rezaei, Milos Radomir Popovic, Uri T. Eden, Milad Lankarany, Ali Yousefi

Keywords: Dynamical Model, State-space Model (SSM), Neural Decoding, Deep neural network (DNN)

Abstract: The state-space models (SSMs) are widely utilized in the analysis of time-series data. SSMs rely on an explicit definition of the state and observation processes. Characterizing these processes is not always easy and becomes a modeling challenge in many instances, such as when the dimension of observed data grows or the observed data distribution deviates from a non-normal distribution. New variants of SSMs try to address these challenges by utilizing deep neural networks (DNNs). Here, we propose a new formulation of SSM for high-dimensional observation processes with heavy-tailed distributions. We call this solution the deep direct discriminative process (D4). D4 utilizes discriminative models like DNN in characterizing the observation process. With this formulation, we bring DNNs' expressiveness and scalability to the SSM formulation that lets us optimally estimate the underlying state process through high-dimensional observation signals. We develop the filter solution for D4 and build a training solution to find the model-free parameters. We demonstrate D4 solutions in simulated and real data such as Lorenz attractors, Langevin dynamics, and rat hippocampus spiking neural data where D4's performance precedes traditional models. D4 can be applied to a broader class of time-series data modeling analysis where the connection between high-dimensional observation and the underlying data generation process is complex and characterizing the conditional distribution of observations is a challenging modeling problem.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Neuroscience and Cognitive Science (e.g., neural coding, brain-computer interfaces)