# NBA-LLM
![main](overview_00.png)
# Abstract
Large language models (LLMs) have demonstrated remarkable zero-shot generalization capabilities, namely the emergence of complex reasoning, such as time series forecasting. However, these LLMs have largely overlooked the quantification of predictive uncertainty, concentrating solely on the accuracy of the deterministic prediction through techniques such as designing prompts and tokenization mechanisms. The predictive uncertainty involves the challenging hallucinations and the overconfidence of LLMs.In this work, we bridge this divide by introducing a novel and model-agnostic Noise informed Bayesian Approximation (NBA) framework that quantifies the uncertainty of frozen LLMs. Furthermore, we derive a mathematical rationale for this uncertainty quantification (UQ). A direct result of this theoretical framework gives us tools to model uncertainty by marginalization of input noise.This mitigates the problem of representing uncertainty in LLM without computational model tuning or test accuracy sacrifice. By studying a diverse range of LLMs across representative real-world and simulated datasets, we investigate the impact of various factors in the UQ in terms of temperature scaling,prediction length, model architecture, and specialized prompting strategies. This framework can serve as a strong baseline and provide insights for UQ in black-box LLMs. 
