Phase-Transitional Scaling

Kalyan Cherukuri, Aarav Lala

Keywords: scaling laws, neural scaling laws, phase scaling laws, transitional

Abstract: We introduce \emph{Phase-Transitional Scaling} (PTS), a falsifiable framework that treats certain emergent abilities of large language models (LLMs) as phase transitions characterized by a sigmoidal response with threshold \(T_K\) and sharpness \(\gamma_K\). We connect this phenomenology to three complementary theoretical perspectives: finite-size mean-field theory, percolation on representational graphs, and noise-activated barrier crossing in training dynamics. Comprehensive experiments across 12 diverse capabilities validate the sigmoid form (47/48 comparisons), demonstrate that \(T_K\) is controlled by data complexity (\(R^2=0.89\)) while \(\gamma_K\) is controlled by training dynamics (\(R^2=0.76\)), and establish universal curve collapse across different architectures (94\% variance explained). PTS provides quantitative, predictive scaling laws that outperform power-law baselines by 4\(\times\) in out-of-sample prediction accuracy.

Submission Number: 204