Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry

Chi-Ning Chou, Hang Le, Yichen Wang, SueYeon Chung

Abstract:

Integrating task-relevant information into neural representations is a fundamental ability of both biological and artificial intelligence systems.
Recent theories have categorized learning into two regimes: the rich regime, where neural networks actively learn task-relevant features, and the lazy regime, where networks behave like random feature models. Yet this simple lazy–rich dichotomy overlooks a diverse underlying taxonomy of feature learning, shaped by differences in learning algorithms, network architectures, and data properties. To address this gap, we introduce an analysis framework to study feature learning via the geometry of neural representations. Rather than inspecting individual learned features, we characterize how task-relevant representational manifolds evolve throughout the learning process. We show, in both theoretical and empirical settings, that as networks learn features, task-relevant manifolds untangle, with changes in manifold geometry revealing distinct learning stages and strategies beyond the lazy–rich dichotomy. This framework provides novel insights into feature learning across neuroscience and machine learning, shedding light on structural inductive biases in neural circuits and the mechanisms underlying out-of-distribution generalization.

Lay Summary:

When you arrive in a new city, how do you learn to get from your hotel to a cozy coffee shop? When researchers train a language model, how does the chatbot learn to solve new math problems? Evidence suggests that both biological neurons in our brain and artificial neurons in machine learning models learn to encode certain “features” relevant to the task they’re trained on—like detecting simple image patterns or representing decision variables.

However, most neurons don’t neatly represent a single, human-interpretable feature. Instead, useful features are often encoded across complex, high-dimensional patterns of activity spread across many neurons. This mixing makes it hard for researchers to understand how neural networks actually learn to solve problems.

In this work, we introduce a new analysis framework to help researchers study feature learning in both biological and artificial systems. Rather than trying to identify features neuron by neuron, we show that you can detect whether a network is learning useful features by examining the geometry of “neural manifolds”—the shapes formed by neural activity patterns. Think of it like organizing your closet: if your clothes are neatly sorted, it’s easier to find what you need. Similarly, when a network learns meaningful features, its internal representations become more organized. This geometric view also helps us distinguish between different learning strategies—like different ways of arranging your wardrobe.

Primary Area: Applications->Neuroscience, Cognitive Science

Keywords: Neural manifold, feature learning, manifold capacity, representational geometry

Submission Number: 7673