Language Bottleneck Models: A Framework for Qualitative Cognitive Diagnosis

ICLR 2026 Conference Submission20513 Authors

Keywords: Cognitive Diagnosis, Knowledge Tracing, Education, Interpretability, LLMs, Language Bottleneck Models

Abstract: Accurately assessing student knowledge is central to education. Cognitive Diagnosis (CD) models estimate student proficiency, while Knowledge Tracing (KT) methods excel at predicting performance over time. However, CD models represent knowledge concepts via quantitative estimates on predefined concepts, limiting expressivity, while KT methods often prioritize accuracy at the cost of interpretability. We propose Language Bottleneck Models (LBMs), a general framework for producing textual knowledge state summaries that retain predictive power. LBMs use an encoder LLM to produce minimal textual descriptions of a student's knowledge state, which a decoder LLM then uses to reconstruct past responses and predict future performance. This natural-language bottleneck yields human-interpretable summaries that go beyond the quantitative outputs of CD models and capture nuances like misconceptions. Experiments show zero-shot LBMs rival state-of-the-art CD and KT accuracy on synthetic arithmetic benchmarks and real-world datasets (Eedi and XES3G5M). We also show the encoder can be finetuned with reinforcement learning, using prediction accuracy as reward, to improve summary quality. Beyond matching predictive performance, LBMs reveal qualitative insights into student understanding that quantitative approaches cannot capture, showing the value of flexible textual representations for educational assessment.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 20513