Neural Representational Geometry of Concepts in Large Language Models

Linden Schrage, Kazuki Irie, Haim Sompolinsky

Keywords: neural manifolds, large language models, linguistic concepts, representational geometry, few-shot learning

TL;DR: We use tools from the geometrical manifold theory to study representations of concepts in large language models.

Abstract: Despite tremendous successes of large language models (LLMs), their internal neural representations remain opaque. Here we characterize the geometric properties of language model representations and their impact on few-shot classification of concept categories. Our work builds on Sorscher et al. (2022)'s theory, previously used to study neural representations in the vision domain. We apply this theory to embeddings obtained at various layers of a pre-trained LLM. We mainly focus on LLaMa-3-8B, while also confirming their applicability to OpenAI's text-embedding-3-large. Our study reveals geometric properties and their variations across layers that are unique to language models, and provides insights into their implications for understanding concept representation in LLMs.

Submission Number: 66