Revisiting the Knowledge Injection Frameworks

Peng Fu, Yiming Zhang, Haobo Wang, Weikang Qiu, Junbo Zhao

Submission Type: Regular Long Paper

Submission Track: NLP Applications

Submission Track 2: Language Modeling and Analysis of Language Models

Keywords: language models, knowledge pruning, large language models for downstream task

Abstract:

In recent years, large language models (LLMs), such as GPTs, have attained great impact worldwide.
However, how to adapt these LLMs to better suit the vertical domain-specific tasks by utilizing external knowledge remains not completely solved.
Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge tuple into the associated text sample.

However, despite the promise, we identify a pivotal problem in this work ubiquitously.
Simply put, we find that injecting unaligned (i.e., random) knowledge tuple into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected.
We therefore take a thorough investigation of this frustrating finding on a variety of related prior work and further provide a chain of potential interpretations for the phenomenon.
Based on all that, we offer a simple remediated technique.
Briefly, the core of this technique roots in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs.
At last, we show that by integrating this technique into most (if not all) knowledge injection frameworks and recent LLMs, it manages to overcome the aforementioned sanity problem and further pushes the boundary of the performance of the domain-adaptive LLMs.

Submission Number: 5767