Spectral learning of Bernoulli linear dynamical systems models for decision-making

Published: 25 Jul 2023, Last Modified: 25 Jul 2023Accepted by TMLREveryoneRevisionsBibTeX
Abstract: Latent linear dynamical systems with Bernoulli observations provide a powerful modeling framework for identifying the temporal dynamics underlying binary time series data, which arise in a variety of contexts such as binary decision-making and discrete stochastic processes such as binned neural spike trains. Here we develop a spectral learning method for fast, efficient fitting of probit-Bernoulli latent linear dynamical system (LDS) models. Our approach extends traditional subspace identification methods to the Bernoulli setting via a transformation of the first and second sample moments. This results in a robust, fixed-cost estimator that avoids the hazards of local optima and the long computation time of iterative fitting procedures like the expectation-maximization (EM) algorithm. In regimes where data is limited or assumptions about the statistical structure of the data are not met, we demonstrate that the spectral estimate provides a good initialization for Laplace-EM fitting. Finally, we show that the estimator provides substantial benefits to real world settings by analyzing data from mice performing a sensory decision-making task.
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: ### Revisions in the updated manuscript posted on 6/9: A summary of the changes since the original submission: * Updated Table 1 of model comparisons to include Poisson LDS from Buesing et. al. (in progress) * Updated Table 2 to include bestLDS computation times * New Supplemental Figure A.1 which characterizes bestLDS performance when there is non-unitized diagonal of Z * New Supplemental Figure A.2 which characterizes the time to run the bestLDS estimator as a function of data size for two different datasets * New Supplemental Figure A.3 which characterizes bestLDS performance against N4SID run directly on the Zs * Numerous edits throughout the text These changes are described in more detail in the responses to reviewers. ### Additional revisions in the updated manuscript posted 6/20: * Updated Table 1 of model comparisons to include additional Poisson LDS results as well as comparisons to linear-Gaussian LDS (this work was still in progress as of the 6/9 submission) ### Additional revisions in the updated manuscript posted 7/21: * Updated Table 1 of model comparisons using the error in the Gain matrix as the comparison metric * Updated text description of Table 1 accordingly * Minor text edits for formatting, clarity, typos, etc. in final review before publication * Checked references for correct formatting
Code: https://github.com/irisstone/bestLDS
Supplementary Material: pdf
Assigned Action Editor: ~Shinichi_Nakajima2
Submission Number: 982