MC^2: A Minimum-Coverage and Dataset-Agnostic Framework for Compositional Generalization of LLMs on Semantic Parsing

ACL ARR 2025 May Submission1334 Authors

Abstract: Compositional generalization is one of the important abilities that large language models (LLMs) need to have for semantic parsing. Previous research typically relies on dataset-specific designs or a large number of samples in demonstrations to improve the compositional generalization of LLMs on semantic parsing. We revisit this issue and find that when the number of samples in a demonstration is limited to a theoretical lower bound for achieving compositional generalization (minimum-coverage), current advanced LLMs cannot arbitrarily achieve good compositional generalization generically on different semantic parsing datasets without dataset-specific designs. To solve this problem, we propose Multi-level Component Composition (MC$^2$), a minimum-coverage and dataset-agnostic framework based on input primitives, which aims to generically help LLMs achieve compositional generalization by selecting and organizing samples from multiple compositional levels that satisfy the primitive coverage. Experiments and analysis show that MC$^2$ can effectively improve the compositional generalization of LLMs on different semantic parsing datasets in the minimum-coverage setting.

Paper Type: Long

Research Area: Semantics: Lexical and Sentence-Level

Research Area Keywords: compositional generalization, semantic parsing

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 1334