Beyond Deep Heuristics: A Principled and Interpretable Orbit-Based Learning Framework

ICLR 2026 Conference Submission12984 Authors

Keywords: Weighted Backward Shift Neural Networks (WBSNNs), orbit dynamics, exact interpolation, operator theory, implicit regularization, interpretable models, data-efficient learning, topology-aware representations, operator learning, infinite-dimensional sequence Banach spaces, cross-modal generalization, lightweight computation

TL;DR: WBSNNs learn a geometric scaffold via exact orbit interpolation, then generalize along it—yielding interpretable, data-efficient, topology-aware predictions from a linear core with orbit-induced nonlinearity and strong cross-modal results.

Abstract: We introduce \emph{Weighted Backward Shift Neural Networks} (WBSNNs), a general-purpose learning paradigm that works across modalities and tasks without per-dataset customization and replaces stacked nonlinearities with structured \emph{orbit dynamics}. WBSNNs comprise a purely linear, operator-theoretic stage that constructs an orbit dictionary that exactly interpolates selected anchors, thereby yielding a faithful geometric scaffold of the dataset, and subsequent predictions reuse this scaffold for generalization by forming data-dependent linear combinations of these orbits—making the model inherently interpretable, as each prediction follows explicit orbit paths on this scaffold, tied to a small, structured subset of the data. While the architecture is built entirely from linear operators, its predictions are nonlinear—emerging from the selection and reweighting of orbit elements rather than deep activation stacks. We further extend this exact-interpolation guarantee to infinite-dimensional sequence Banach spaces (e.g., $\ell^p$, $c_0$), positioning WBSNNs as suitable for operator-learning problems in these spaces. WBSNNs demonstrate robust generalization; we provide a formal proof that their structure induces implicit regularization in stable dynamical regimes—regimes which, in most of our experiments, emerged automatically without any explicit penalty or constraint on the core orbit dynamics—and consistently match or outperform strong baselines across different tasks involving nontrivial manifolds, high-dimensional speech and text classification, sensor drift, air pollution forecasting, financial time series, image recognition with compressed features, distribution shift and noisy low-dimensional signals—often using as little as $1$% of the training data to build the orbit dictionary. Despite its mathematical depth, the framework is computationally lightweight and requires minimal engineering to achieve competitive results, particularly in noisy low-dimensional regimes. These findings position WBSNNs as a highly interpretable, data-efficient, noise-robust, and topology-aware alternative to conventional neural architectures.

Primary Area: learning on time series and dynamical systems

Submission Number: 12984