everyone
since 13 Oct 2023">EveryoneRevisionsBibTeX
Large Language Models (LLMs) have quickly become popular tools for recalling information; however, the specific mechanisms they utilize to encode and store vast information within their parameters is not well understood. As a step towards improving interpretability and reliability of LLMs, we develop AttributeLLaMA, which involves fine-tuning LLaMA models to enable them to attribute their responses. By utilizing a mere 100 expert-annotated attribution examples, we are able to achieve this capability. Our experimental studies demonstrate that these attributions significantly improve the performance of a strong LLM on downstream tasks such as MMLU and StrategyQA by more than 3.5% and 4.0%, respectively. Furthermore, our analyses on the attributions obtained from AttributeLLaMA reveal the remarkable memorization capabilities of LLMs