Quantum-RAG and PunGPT2: Advancing Low-Resource Language Generation and Retrieval for the Punjabi Language

JASKARANJEET SINGH, Rakesh Thakur

Keywords: Low-resource languages, PunGPT2, RAG, Pun-RAG, Pun-Instruct, QLoRA

TL;DR: We introduce PunGPT2, the first open-source Punjabi LLM suite with quantum-inspired retrieval, outperforming multilingual baselines and advancing NLP for low-resource languages.

Abstract:

Retrieval-augmented models rely heavily on similarity functions such as cosine or dot-product, which often under-represent fine-grained semantic cues—
especially in low-resource languages with sparse contextual coverage. We introduce Quantum-RAG, a phase-augmented retrieval mechanism that extends classical similarity with learnable interference terms, enabling richer relevance estimates with minimal computational overhead. Quantum-RAG generalises cosine
similarity and can be integrated into any dual-encoder or dense retrieval setup. To
demonstrate its effectiveness, we pair it with a new Punjabi generative model suite
(PunGPT2, Pun-RAG, Pun-Instruct) trained on a curated 35GB corpus. Across retrieval metrics and generation benchmarks, Quantum-RAG yields substantial improvements over FAISS (e.g., +7.4 Recall@10) and over multilingual LMs on PunjabiEval and FLORES-200. We additionally report small-scale Hindi and Bangla
experiments showing cross-lingual gains (+3–5 Recall@10). All datasets, training
configurations, and evaluation pipelines are released for full reproducibility

Primary Area: generative models

Submission Number: 20713