MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databases

ACL ARR 2025 February Submission803 Authors

Abstract:

The improvement in translating natural language to structured query language (SQL) can be attributed to the advancements in large language models (LLMs). Open-source LLMs, tailored for specific database dialects such as MySQL, have shown great performance. However, cloud service providers are looking for a unified database manager service (e.g., Cosmos DB from Azure, Amazon Aurora from AWS, Lindorm from AlibabaCloud) that can support multiple dialects.
This requirement has led to the concept of multi-dialect query generation, which presents challenges to LLMs. These challenges include syntactic differences among dialects and imbalanced data distributions across them.
To address these issues, we propose MoMQ, a novel Mixture-of-Experts-based multi-dialect query generation framework across both relational and non-relational databases.
MoMQ incorporates dialect-specific expert groups to capture syntax features of individual dialects while minimizing cross-dialect interference in query generation. Additionally, we propose a multi-level routing mechanism enhanced by Dialect Router Loss (DRL) and a shared expert group architecture, facilitating common knowledge transfer from high-resource dialects to low-resource ones.
Furthermore, we have developed a high-quality multi-dialect query generation benchmark that covers relational and non-relational databases such as MySQL, PostgreSQL, Cypher for Neo4j, and nGQL for NebulaGraph.
Extensive experiments have shown that MoMQ performs effectively and robustly, even in resource-imbalanced scenarios.

Paper Type: Long

Research Area: Generation

Research Area Keywords: model architectures, text-to-text generation, data-to-text generation, multilingualism

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources

Languages Studied: English

Submission Number: 803