A Horizon-Dependent Intrinsic-Dimension Theory of Scaling for Biological Forecasting

Bryan Cheng, Austin Jin, Jasper Zhang, Arnav Pemmaraju, Brendan Lo, Joshua Chang

Keywords: Neural scaling laws, Lyapunov exponents, single-cell foundation models, perturbation forecasting, Kolmogorov–Sinai entropy, dynamical systems, manifold dimension, irreducibility floor, information bottleneck, scale-or-stop

TL;DR: Predicting further ahead is fundamentally harder: the effective manifold dimension grows with horizon × chaos rate, capping scaling gains. Explains scGPT/Geneformer saturation and X-Cell-Ultra's clean α=0.32.

Abstract: Neural scaling laws derive exponents from the intrinsic dimension of static data manifolds, but most consequential prediction tasks — weather, climate, single-cell perturbation forecasting — are dynamical. We propose HORIZON, a horizondependent extension of the Sharma–Kaplan/Bahri scaling law in which the predictively-sufficient statistic at horizon h lives on a manifold of dimension d ⋆ (h) = d0 + R h 0 Λ+(τ ) dτ / log(1/ε), with Λ+ the local sum of positive Lyapunov exponents. The resulting law L(N, D; h) = L∞(h) + A N −4/d⋆ (h) + B D−2/d⋆ (h) reduces to Sharma–Kaplan as h → 0 and predicts that interventional pretraining truncates the Lyapunov integral. We test HORIZON empirically across 300 training runs on three chaotic systems with directly-measured Lyapunov spectra. A bottleneck-controlled autoencoder confirms the theory’s central interpretation: at bottleneck width k = 1, the observed exponent α = 4.52 matches the prediction α = 4/k within 13%, while the irreducibility floor L∞(k) drops monotonically with transitions co-located with d ⋆ (h) across all three systems (Lorenz-63, Lorenz-96 N=10, Lorenz96 N=20). HORIZON yields a closed-form scale-or-stop decision rule that classifies the surveyed observational scFMs (scGPT, GeneformerV1/V2, scFoundation, C2S-Scale) as past their predicted ceiling and X-Cell-Ultra as on-ceiling — matching empirical saturation patterns in the literature.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 168