Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models

ACL ARR 2025 May Submission4357 Authors

Abstract:

Seminal work by Huebner et al. (2021) showed that language models (LMs) trained on English Child-Directed Language (CDL) can outperform LMs trained on an equal amount of adult-directed text like Wikipedia.
However, it remains unclear whether these results generalize across languages, architectures, and evaluation settings.
We test this by comparing models trained on CDL vs. Wikipedia across two LM objectives (masked and causal), three languages (English, French, German), and three syntactic minimal pair benchmarks.
Our results on these benchmarks show inconsistent benefits of CDL, which in most cases is outperformed by Wikipedia models.
We then identify various shortcomings in these benchmarks, and introduce a novel testing methodology, FIT-CLAMS, which uses a frequency-controlled design to enable balanced comparisons across training corpora.
Through minimal pair evaluations and regression analysis we show that training on CDL does not yield stronger generalizations for acquiring syntax and highlight the importance of controlling for frequency effects when evaluating syntactic ability.

Paper Type: Long

Research Area: Linguistic theories, Cognitive Modeling and Psycholinguistics

Research Area Keywords: linguistic theories, multilingual evaluation, evaluation methodologies

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English, French, German

Submission Number: 4357