Learning Compositional Hierarchies using Curriculum Learning for Compositional Generalization

Adam Dahlgren

Keywords: compositional generalization, curriculum learning, developmental psychology, concept learning

TL;DR: What can we learn from developmental psychology to create curricula for compositional generalization?

Abstract: Compositional generalization is an integral part of human intelligence. We describe insights from developmental psychology in how humans acquire new words and associated concepts. One key component in such experiments is using pseudowords to avoid pollution from previous experience. Current machine learning approaches for language differ vastly from the process outlined by experiments with humans. We argue for a research direction investigating how curriculum learning and concept learning using pseudowords can affect central capabilities such as compositional generalization. Previous work mostly focus on learning hierarchies of concepts with only one layer of abstraction. Relying on knowledge of more complex hierarchies can help language models learn more efficiently, but also achieve internal structure better suited for compositional generalization.