A Learning Law: Generalization via Geometric Complexity and Algebraic Capacity

TMLR Paper6574 Authors

Abstract: Modern machine learning systems achieve strong performance but remain data-hungry and opaque. We propose the \emph{Learning Law}, which asserts that effective learning follows the order \emph{form $\rightarrow$ law $\rightarrow$ data $\rightarrow$ understanding}. We formalize this by separating geometry discovery, law formation, and data calibration. The first stage learns a latent manifold with controlled intrinsic dimension and smoothness. The second restricts predictors to an algebraically constrained law space on this geometry. The third calibrates these laws on finite labeled data. We derive a Geometry–Algebra Generalization Bound showing that population risk depends on geometric complexity $\mathcal{C}(\phi)$ and algebraic capacity $\mathcal{A}(g)$, rather than raw parameter count, yielding intrinsic sample-efficiency advantages for geometry-first learning. A two-stage V-GIB implementation confirms these predictions on CIFAR-10 and a tabular classification task. Geometry-first pretraining lowers intrinsic dimension, improves low-label test accuracy, and outperforms data-first baselines once training stabilizes, with ablations isolating the roles of smoothness and intrinsic-dimension control.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Sebastian_Goldt1

Submission Number: 6574