Spectral learning of Bernoulli linear dynamical systems models for decision-making

Iris R Stone, Yotam Sagiv, Il Memming Park, Jonathan W. Pillow

Abstract:

Latent linear dynamical systems with Bernoulli observations provide a powerful modeling framework for identifying the temporal dynamics underlying binary time series data, which arise in a variety of contexts such as binary decision-making and discrete stochastic processes
such as binned neural spike trains. Here we develop a spectral learning method for fast, efficient fitting of probit-Bernoulli latent linear dynamical system (LDS) models. Our approach extends traditional subspace identification methods to the Bernoulli setting via a transformation of the first and second sample moments. This results in a robust, fixed-cost estimator that avoids the hazards of local optima and the long computation time of iterative fitting procedures like the expectation-maximization (EM) algorithm. In regimes where data is limited or assumptions about the statistical structure of the data are not met, we demonstrate that the spectral estimate provides a good initialization for Laplace-EM fitting. Finally, we show that the estimator provides substantial benefits to real world settings by analyzing data from mice performing a sensory decision-making task.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

Revisions in the updated manuscript posted on 6/9:

A summary of the changes since the original submission:

• Updated Table 1 of model comparisons to include Poisson LDS from Buesing et. al. (in progress)
• Updated Table 2 to include bestLDS computation times
• New Supplemental Figure A.1 which characterizes bestLDS performance when there is non-unitized diagonal of Z
• New Supplemental Figure A.2 which characterizes the time to run the bestLDS estimator as a function of data size for two different datasets
• New Supplemental Figure A.3 which characterizes bestLDS performance against N4SID run directly on the Zs
• Numerous edits throughout the text

These changes are described in more detail in the responses to reviewers.

Additional revisions in the updated manuscript posted 6/20:

• Updated Table 1 of model comparisons to include additional Poisson LDS results as well as comparisons to linear-Gaussian LDS (this work was still in progress as of the 6/9 submission)

Additional revisions in the updated manuscript posted 7/21:

• Updated Table 1 of model comparisons using the error in the Gain matrix as the comparison metric
• Updated text description of Table 1 accordingly
• Minor text edits for formatting, clarity, typos, etc. in final review before publication
• Checked references for correct formatting

Code: https://github.com/irisstone/bestLDS

Assigned Action Editor: Shinichi Nakajima

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 982