Talking with Oompa Loompas: A Novel Framework for Evaluating Linguistic Acquisition of LLM Agents

Anshika Krishnatray, Sankalp Tattwadarshi Swain, Dhruv Kumar, Jagat Sesh Challa

Keywords: LLM evaluation, Language acquisition, Adaptive feedback

Abstract: Existing evaluation studies on linguistic competence of large language models (LLMs) have focused primarily on vocabulary learning, morphological rule induction, syntactic generalization, pragmatic inference, and cross-linguistic transfer. However, none assess whether LLMs can acquire a language through pattern recognition and interactive feedback, a central feature of human language acquisition. We propose a novel experimental framework in which an LLM is evaluated on its ability to acquire and use a newly constructed language (Tinkatongue) in conversation with a bot that understands only Tinkatongue. Our findings show that LLMs fail to establish a conversation within 100 responses, yet they adopt distinct strategies that mirror human approaches to language learning. The results suggest a new direction for evaluation benchmarks and open pathways to model designs that learn more effectively from interactive feedback.

Submission Number: 375