Reconstructing Reality: A Collective Social Simulation of Belief Propagation from Distributed Evidence

Keywords: social simulations, LLMs, knowledge reconstruction

TL;DR: Collective social simulation on reconstructing knowledge using LLMs

Abstract: We introduce a controlled, abstract multi-agent simulation framework for studying how a population of autonomous agents—each initialized with small, overlapping and noisy subsets of facts—can reconstruct a latent ground-truth knowledge base through local interactions. Agents iteratively share high-confidence items and update belief scores by aggregating received evidence. We evaluate three agent families (Heuristic, Homogeneous LLM-based, and Heterogeneous LLM-based) on a family-relationship domain across a parameter sweep (population size, communication bandwidth, confidence thresholds, sharing strategies, and number of rounds). Our experiments show that a rule-based Heuristic configuration attains near-perfect precision and high F1 ($0.943$), while both LLM-based configurations (Homogeneous and Heterogeneous) struggle to reach accurate consensus (mean F1 $\approx0.28$). We identify a strong effect of sharing strategy (``highest\_confidence'' improves non-heuristic performance substantially) and systematic weaknesses on negative and marriage facts. We analyze convergence behavior, noting that very few runs (2.6\%) converge naturally, with most terminating at the round limit. The code and data can be found \href{https://github.com/Mystic-Slice/Agents4Science-Simulation}{here}.

Submission Number: 180