The Concept Percolation Hypothesis: Analyzing the Emergence of Capabilities in Neural Networks Trained on Formal Grammars

Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka

Keywords: Emergence, Safety, Percolation, Capabilities

Abstract: We analyze emergence of capabilities as a function of learning time, i.e., learning curve analysis. Training models on a well-defined, synthetic context-sensitive formal language, we find the existence of precise phases that separate the learning dynamics. Motivated by our results, we propose a qualitative theory grounded in the process of graph percolation that describes a mechanistic basis for how capabilities may be emerging in neural networks as they are trained on increasingly larger datasets.

Submission Number: 107