Graph Signal Processing Meets Mamba2: Adaptive Filter Bank via Delta Modulation

ICLR 2026 Conference Submission15463 Authors

Keywords: State Space Model, Mamba, Graph Signal Processing, Adaptive Filter Bank

TL;DR: HADES reinterprets Mamba2 as a graph-based adaptive filter bank, achieving efficient and interpretable sequence modeling with fewer parameters.

Abstract: State-space models (SSMs) offer efficient alternatives to attention with linear-time recurrence. Mamba2, a recent SSM-based language model, uses selective input gating and a multi-head structure, enabling parallel computation and strong benchmark performance. However, its multi-head recurrence operates independently without structured utilization or analysis. In this work, we propose a novel method called **H**ierarchical **AD**aptive filter bank for **E**fficient **S**SMs (*HADES*), a Graph Signal Processing (GSP)-inspired framework that reinterprets Mamba2 as an adaptive filter bank on a line graph. Our hierarchical architecture introduces two filter types: shared filters for global low-pass behavior and expert filters for local high-pass behavior, achieved through structured bias on the parameter $\Delta$. *HADES* achieves comparable performance to baseline models including Mamba2 across various benchmarks in language modeling, commonsense reasoning, and long-context retrieval, while using only **58.9%** of the original parameters. In this regard, *HADES* bridges GSP and neural sequence modeling, enabling efficient, hierarchical, and interpretable filtering within state-space models.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 15463