HARBOR: Hierarchical Abduction with Bayesian Orchestration for Reliable Probability Inference in Large Language Models

Wentao Qiu, Guanran Luo, Zhongquan Jian, Jingqi Gao, Meihong Wang, Qingqiang Wu

Keywords: Probability Estimation, Large Language Model

Abstract:

A central challenge in large-scale decision-making under incomplete information is the estimation of reliable probabilities. Prior work has employed Large Language Models (LLMs) to generate relevant factors and provide initial, coarse-grained probability estimates. These methods typically utilize an LLM for forward abduction to generate factors, with each factor constrained to two mutually exclusive attributes. A Naïve Bayes model is then trained on combinations of these factors to provide more accurate probabilities. However, this approach often yields a sparse factor space, resulting in "unknown" predictions where the model fails to produce an output. Naively increasing the number of factors to densify the space not only introduces statistical noise but also violates the Naïve Bayes independence assumption, ultimately compromising the stability and reliability of the estimates.
To address these limitations, we propose Harbor, a novel inference framework that orchestrates aggregated Bayesian inference over a hierarchically structured factor space. Harbor first constructs a dense, structured factor space through iterative generation and hierarchical clustering. It then performs context-aware mapping using retrieval and refinement operations on this hierarchy to reduce "unknown" predictions. Finally, Harbor extends Naïve Bayes by incorporating a Causal Bayesian Network to model latent dependencies, thereby relaxing the strict independence assumption. Experiments show that Harbor substantially reduces "unknown'' predictions and yields more reliable probabilities than direct LLM baselines, achieving state-of-the-art performance with significantly reduced time and token overhead.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 6823