Spoken Language Understanding Evaluations for Home-Based Basic Math Learning
Keywords: Spoken Dialogue Systems (SDS), Spoken Language Understanding (SLU), Natural Language Understanding (NLU), Automatic Speech Recognition (ASR), NLP in Education, Early Math Learning
TL;DR: Spoken Language Understanding (SLU) pipeline evaluations with cascaded ASR and NLU modules for kids learning basic math in home settings.
Abstract: Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores the Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our Kid Space home deployment data with children going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the Google Cloud and the recent open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children's speech for math learning in authentic homes.
Submission Number: 41
Loading