Emergence, pretraining loss and associative recall: a toy model

Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai

Keywords: emergence, time-series, toy models, interpretability

TL;DR: To study emergence in LLM-style neural networks, we introduce a new family of toy problems that combine features of linear-regression style continuous in-context learning (ICL) with discrete associative recall.

Abstract: To study emergence in LLM-style neural networks, we introduce a new family of toy problems that combine features of linear-regression style continuous in-context learning (ICL) with discrete associative recall --- specifically symbolically labeled interleaved observations from randomly drawn deterministic linear dynamical systems. We pretrain transformer models on sample traces from this toy, and explore the idea that the emergence of an ability is largely a function of the pretraining loss. During training, this toy model exhibits the emergence of at least three different abilities, and we use simple out-of-distribution experiments to show how some of these abilities seem to completely ignore what feels to a human as being very salient context.

Submission Number: 56