Using What You Know: Understanding the Role of Supervised Learning in Using Semantic Knowledge

Zach Studdiford, Gary Lupyan

Track: long paper (up to 10 pages)

Domain: cognitive science

Abstract: Self-supervised learning, in which a system learns by minimizing its own prediction errors, is immensely powerful. Modern LLMs are often used as an example of just how much structure it is possible to learn from self-supervision alone. However, just as human cognitive development relies on structured feedback that is parenting and education, it is worth asking how supervised learning augments what LLMs can do. After all, no consumer-facing LLMs are purely self-supervised. We systematically compare purely self-supervised base LLMs to models further refined with supervised training and find that although self-supervision is sufficient for learning the basic knowledge needed to answer our queries, base models fail to use the knowledge in appropriate ways. Further examination of the internal states of the LLMs reveals that models exposed to supervised learning learn to align their semantic representations with the prompt in a way that enables coherent responses.

Presenter: ~Zach_Studdiford1

Submission Number: 101