Talking with Oompa Loompas: A novel framework for evaluating linguistic acquisition of LLMs

Sankalp Tattwadarshi Swain, Anshika Krishnatray, Dhruv Kumar, Jagat Sesh Challa

Keywords: Language Acquisition, Adaptive Feedback, Linguistic Competence

TL;DR: Can LLM agents talk in Tinkatongue with Oompa Loompas?

Abstract: Existing evaluation studies on linguistic competence of large language models have focused primarily on vocabulary learning, morphological rule induction, syntactic generalization, pragmatic inference, and cross-linguistic transfer. However, none assess whether LLMs can acquire a language through pattern recognition and interactive feedback, a central feature of human language acquisition. We propose a novel experimental framework in which an LLM is evaluated on its ability to acquire and use a newly constructed language in conversation (Oompa-Loompish) with a bot that understands only Oompa-Loompish. Our findings show that LLMs fail to establish a conversation within 100 responses, yet they adopt distinct strategies that mirror human approaches to language learning. The results suggest a new direction for evaluation benchmarks and open pathways to model designs that learn more effectively from interactive feedback.

Submission Number: 99