Adversarial Databases Improve Success in Retrieval-based Large Language Models

Sean Wu, Michael Koo, Li Yo Kao, Andy Black, Lesley Blum, Fabien Scalzo, Ira Kurtz

Keywords: Retrieval Augmented Generation, Adversarial Databases, Large Language Models, Medicine

TL;DR: We reveal that RAG on adversarial databases can significantly improve the MCQ test taking ability of LLMs, suggesting a need for further exploration into the mechanism(s) involved and the generalizability of this new phenomenon we have discovered.

Abstract: Retrieval-Augmented Generation (RAG) is utilized to enhance large language model (LLM) performance by leveraging external knowledge databases. While it is generally believed that adversarial databases should negatively impact RAG's effectiveness, we tested this assumption for the first time in the context of the medical subspecialty field of Nephrology. We used several open-source LLMs, including Llama 3, Phi-3, Mixtral 8x7b, Zephyr$\beta$, and Gemma 7B Instruct in a zero-shot RAG pipeline, incorporating both relevant databases (nephSAP and UpToDate) and adversarial databases (Bible and Random Words). Suprisingly, our results show that adversarial Bible and Random Words databases significantly improved Nephrology multiple choice question (MCQ) test-taking ability of specific LLMs. Utilizing DistilBERT's attention outputs, we provide evidence that adversarial databases can potentially affect LLM performance through changes in attention. Our findings highlight the need for further research into the mechanism(s) and generality of the effect of adversarial databases on LLM performance that we have discovered.

Submission Number: 10