SaFARi: State-Space Models for Frame-Agnostic Representation

Hossein Babaei, Mel White, Sina Alemohammad, Richard Baraniuk

Abstract:

State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data.
However, to date, only a few polynomial bases have been explored for this purpose, and the state-of-the-art implementations were built upon the best of a few limited options. In this paper, we present a generalized method for building an SSM with any frame or basis, rather than being restricted to polynomials.
This framework encompasses the approach known as HiPPO, but also permits an infinite diversity of other possible "species" within the SSM architecture. We dub this approach SaFARi: SSMs for Frame-Agnostic Representation.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=SlCMxNfB1W

Changes Since Last Submission:

This version represents the camera-ready submission of the paper. All author names have now been added, and the manuscript has been fully deanonymized. The previously anonymous code link has been replaced with the official GitHub repository.

All temporary formatting used to highlight changes (e.g., colored text) has been reverted to standard black text for consistency. Several minor typographical errors have also been corrected.

In response to reviewer feedback, we have added a standard deviation analysis. The corresponding results are presented as a new table in Appendix A.9, accompanied by a brief discussion.

Video: https://github.com/echbaba/safari-ssm/blob/main/Safari%20ssm%20short%20presentation.mp4

Code: https://github.com/echbaba/safari-ssm

Assigned Action Editor: ~Grigorios_Chrysos1

Submission Number: 5222