A Causal Study on The Learnability of Formal Languages

Vésteinn Snæbjarnarson, Josef Valvoda, Anej Svete, Brian DuSell, Ryan Cotterell

Keywords: Causality, language models, formal languages

TL;DR: We introduce a new semiring for causaly intervening on formal automata, and use the resulting datasets to evaluate Transformer LMs.

Abstract: Understanding the limitations of neural language models is crucial for knowing what such models are capable of and how they can be used safely. A popular approach to analyzing formal limitations takes the form of training models on formal languages, and studying what aspects of the languages affect model performance. Formal languages can, for instance, be designed using manually constructed grammars or randomly sampled by sampling some type of automata. This provides the researcher with unique control over the features of the language of interest. In this paper, we provide an even more fine-grained approach to targeted model evaluation. We develop a method for controlling specific \emph{string} features, on the corpus level, in the language of a given automaton. This gives us control over properties such as symbol frequencies while keeping everything else intact, enabling a causal study of their importance. To describe our framework formally, we turn to \emph{semirings} and introduce finite state automata over a novel---counting---semiring. We devise algorithms that enable string sampling under varying degrees of interventions and demonstrate the utility of our method through several examples showing how targeted interventions over transition, symbol, and state frequencies can be performed. We then train Transformer and LSTM language models on languages under varying degrees of interventions. Our fine-grained analysis allows us to show that different mechanisms influence the learning behavior of these two architectures.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10695