Optimising Screen-Based Learning for Children with Neuropersonalised Acoustic Ostensive Signals

Lee Kean Mun, Priyadharshini Santhanakrishnan, Johann Kay Ann Tan, Manoj Ramanathan, Bisio Isabella Sole, Kwa En Hui Anna, Joan Kailin Ng, Tan Yu Wei Russell, Teo Hui Ling Rachel, Victoria Leong

Track: Full paper

Keywords: Digital Learning, Neuropersonalisation, Early Language Learning, Ostensive Signals, Video Deficit Effect, Infant-Directed Speech

Abstract: Current digital learning tools for young children frequently rely on pre-recorded vocalisations or model their character’s vocalisations based on acoustic properties that were derived from face-to-face interactions. However, it is known in developmental psychology that social contingency and reciprocal interaction are crucial. The “video deficit” effect suggests that screen-based stimulus presentation typically fails to engage learning mechanisms in the developing brain, largely because these interactions remain passive, lacking the necessary back-and-forth modulation that fosters effective learning. In our first study, we compare children’s language learning outcomes during live video calls and face-to-face sessions in order to identify statistical characteristics of key ostensive (teaching) signals that vary in these conditions and explain differences in learning. In a second ongoing experiment, we directly manipulate speech rate and introduce a background song, which serves as an entrainment stimulus in accordance with the child’s individual peak alpha frequency in order to personalise the learning context. Our results from the first study reveal distinct modifications in caregivers’ speech patterns, particularly in their speech rate and vowel hyperarticulation across settings, indicating that mothers spontaneously adjust their Infant-Directed Speech (IDS) to compensate for reduced physical immediacy during video calls. Meanwhile, the second study is on-going. These findings suggest that early digital learning programmes should incorporate learning cues that are naturally present in real-time, contingent interactions. Additionally, by capturing acoustic features specific to video-based exchanges, next-generation learning tools may more effectively engage learners, mitigating the video deficit effect and supporting infants’ language acquisition in screen-based contexts.

Submission Number: 7