Learning and Injecting Sparse Concept Graphs for Interpretable Clinical Large Language Modeling

ACL ARR 2026 January Submission9560 Authors

Keywords: knowledge injection, LLM, matrix normal graphical model, medical text analysis, nonparametric statistics

Abstract: Large language models (LLMs) achieve strong results across NLP tasks, but they often struggle to exploit structured domain knowledge in specialized settings such as clinical text, where medical concepts are linked by sparse conditional dependencies. To address this challenge, we present \textbf{GRIAN} (Graph-Regularized and Injected Adaptation Network), which treats the existence of a sparse concept dependency graph as an explicit prior during LLM adaptation. Rather than learning a graph separately, GRIAN integrates graph recovery into training by augmenting the large language modeling objective with a nonparanormal matrix-normal graphical-model loss that jointly estimates sparse precision matrices while optimizing the LLM. The graph-structured term regularizes the model toward parsimonious conditional dependencies, and is further complemented by a Laplacian smoothness regularizer that aligns concept embedding geometry with the emerging structure via parameter-efficient LoRA updates. For downstream prediction, we encode query-conditional induced subgraphs with a graph attention network. Then we inject graph evidence into Transformer attention, enabling structure-grounded and more interpretable reasoning over clinical text. Experiments on a Chinese knee-joint electronic medical record dataset and a medical abstract dataset show consistent improvements over LLM baselines, highlighting the benefits of jointly regularized graph-structure adaptation for reliable and interpretable clinical text modeling.

Paper Type: Long

Research Area: Clinical and Biomedical Applications

Research Area Keywords: Machine Learning for NLP, NLP Applications

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: Chinese, English

Submission Number: 9560