Neural Diversity Regularizes Hallucinations in Language Models

Kushal Chakrabarti, Nirmal Balachundhar

Keywords: hallucination suppression

TL;DR: Neural diversity—de-correlated parallel streams—provably and empirically reduces hallucinations in LLMs at fixed parameter and data budgets.

Abstract: Language models continue to hallucinate despite increases in parameters, compute, and data. We propose *neural diversity* — decorrelated parallel representations — as a principled mechanism that reduces hallucination rates at fixed parameter and data budgets. While existing mitigation strategies largely target accuracy, we provide the first formal tail bounds for hallucination probability in ensembled language models, reframing it as a second-moment reliability problem and *explaining 96.2% of empirical reliability variation* seen across parallel configurations. We introduce ND-LoRA (Neural Diversity Low-Rank Adaptation), combining parallel LoRA adapters with Barlow Twins regularization, and *reduce hallucinations by up to 25.6% (and 14.6% on average)* while preserving general accuracy. Ablations show LoRA adapters and regularization act synergistically, causal interventions prove neurodiversity as the mediating factor and correlational studies indicate scale: a 0.1\% neural correlation increase is associated with a 3.8\% hallucination increase. Finally, task-dependent optimality emerges: different tasks require different optimal amounts of neurodiversity. Together, our results highlight neural diversity as a third axis of scaling — orthogonal to parameters and data — to *improve the reliability of language models at fixed budgets*.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 22209