A Small Math Model: Recasting Strategy Choice Theory in an LLM-Inspired Architecture

Roussel Rahman, Jeff Shrager

Keywords: Developmental Modeling, Transformers, Mathematical Reasoning, Strategy Choice

TL;DR: We recast the Strategy Choice Theory that explains children's arithmetic learning, in a neural-network-based architecture analogous to LLMs, to explore and explain how mathematical reasoning and strategy development emerge in artificial minds.

Abstract: Strategy Choice Theory (SCT) explains important aspects of children's arithmetic learning based upon principles including learning from developmentally naturalistic data, probabilistic representation, confidence-based retrieval, and the phase-like importance of scaffolding strategies, such as finger-counting. Here we recast SCT as a "Small Math Model" (SMM), employing a neural-network-based architecture analogous to LLMs. The SMM extends SCT to include counting practice, symbol (number) embedding, and gated attention. Similar to earlier work, the SMM demonstrates constructive and destructive interference between counting and addition, and the ``wave-like'' use of finger-counting as sum recall improves. We plan to extend the SMM to later aspects of the decades-long SCT program, including adaptive strategy choice and eventually strategy discovery, providing a unified platform to investigate the understanding of numerical characteristics and relationships essential for mathematical reasoning -- as it can emerge in LLM-based agents.

Submission Number: 67