SaFARi: State-Space Models for Frame-Agnostic Representation

TMLR Paper5222 Authors

Abstract:

State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data.
However, to date, only a few polynomial bases have been explored for this purpose, and the state-of-the-art implementations were built upon the best of a few limited options. In this paper, we present a generalized method for building an SSM with any frame or basis, rather than being restricted to polynomials.
This framework encompasses the approach known as HiPPO, but also permits an infinite diversity of other possible "species" within the SSM architecture. We dub this approach SaFARi: SSMs for Frame-Agnostic Representation.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=SlCMxNfB1W

Changes Since Last Submission: This paper originates from an earlier submission that was desk-rejected due to length, and breaching anonymity. It has since been shortened and substantially rewritten to focus only on the theoretical components. The current version includes a new abstract and reorganized exposition. It is a self-contained and distinct manuscript.

Assigned Action Editor: ~Grigorios_Chrysos1

Submission Number: 5222