MODE: Learning compositional representations of complex systems with Mixtures Of Dynamical Experts

ICLR 2026 Conference Submission19373 Authors

Keywords: dynamical systems, mixture of experts, interpretable machine learning, clustering, time series forecasting, systems biology, single cell RNA sequencing

TL;DR: MODE (Mixture Of Dynamical Experts): a graphical modeling framework which decomposes complex dynamics into interpretable components, enabling the unsupervised discovery of behavioral regimes and long-term forecasting across regime transitions.

Abstract: Dynamical systems in the life sciences are often composed of complex mixtures of overlapping behavioral regimes. For example, cellular subpopulations may shift from cycling to equilibrium dynamics or branch towards different developmental fates. The transitions between these regimes can appear noisy and irregular, posing a serious challenge to traditional, flow-based modeling techniques which assume locally smooth dynamics. To address this challenge, we propose MODE (Mixture Of Dynamical Experts), a graphical modeling framework whose neural gating mechanism decomposes complex dynamics into sparse, interpretable components, enabling both the unsupervised discovery of behavioral regimes and accurate long-term forecasting across regime transitions. Crucially, because agents in our framework can jump to different governing laws, MODE is especially tailored to the aforementioned noisy transitions. We evaluate our method on a battery of synthetic and real datasets from computational biology. First, we systematically benchmark MODE on an unsupervised classification task using synthetic dynamical snapshot data, demonstrating its favorable performance versus standard clustering methods, especially in few-sample settings. Next, we show how MODE outperforms a neural ODE baseline on challenging forecasting tasks which simulate key cycling and branching processes in cell biology. Finally, we deploy our method on human, single-cell RNA sequencing data and show that it can accurately distinguish proliferation from differentiation dynamics. Furthermore, we show how MODE’s dynamic mixture weights during forecasting can help predict when developing cells will commit to their ultimate fate, a key outstanding challenge in computational biology. Overall, MODE provides a straightforward, interpretable approach to compositional modeling of complex systems in the life sciences and beyond.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 19373