Infinite-Dimensional HiPPO Provides an Explicit Formula for LSSLs

Atsushi Takabatake, Takaharu Yaguchi

Keywords: state space model, HiPPO matrix, representation of solution

TL;DR: We show that an infinite-dimensional LSSL with HiPPO admits an explicit solution.

Abstract: Recently, models for sequence processing, such as large-scale language models, have become increasingly important. Among these, the Linear State Space Layer (LSSL) has been proposed as a fast sequence-processing model. It is also known that using the HiPPO matrix in these LSSLs improves their performance. In this paper, we extend the HiPPO matrix to an operator on function spaces. Furthermore, we show that the resulting infinite-dimensional LSSL with HiPPO admits an explicit solution.

Submission Number: 25