Content Knowledge Identification with Multi-agent Large Language Models (LLMs)

Kaiqi Yang, Yucheng Chu, Taylor Darwin, Ahreum Han, Hang Li, Hongzhi Wen, Yasemin Copur-Gencturk, Jiliang Tang, Hui Liu

Abstract: Teachers’ mathematical content knowledge (CK) is of vital importance and need in teacher professional development (PD) programs. Computer-aided asynchronous PD systems are the most recent proposed PD techniques. However, current automatic CK identification methods face challenges such as diversity of user responses and scarcity of high-quality annotated data. To tackle these challenges, we propose a Multi-Agent LLMs-based framework, LLMAgent-CK, to assess the user responses’ coverage of identified CK learning goals without human annotations. Leveraging multi-agent LLMs with strong generalization ability and human-like discussions, our proposed LLMAgent-CK presents promising CK identifying performance on a real-world mathematical CK dataset MaCKT.