Context-Aware Reasoning On Parametric Knowledge for Inferring Causal Variables

ACL ARR 2025 February Submission4647 Authors

Abstract: Scientific discovery catalyzes human intellectual advances, driven by the cycle of hypothesis generation, experimental design, evaluation, and assumption refinement. This process, while crucial, is expensive and heavily dependent on the domain knowledge of scientists to generate hypotheses. Recent work shows the potential of LLMs in assisting in scientific discovery and inferring causal relationships. Building on this, we introduce a novel task where the input is a partial causal graph with missing variables, and the output is a hypothesis about the missing variables. To evaluate this, we design a benchmark with varying difficulty levels. We show the strong ability of LLMs to hypothesize the mediation variables between a cause and its effect. In contrast, they underperform in hypothesizing the cause and effect variables themselves. Unlike simple knowledge memorization of fixed associations, this task requires the LLM to reason according to the context of the entire graph. This enables researchers to identify new variables of interest during the evolving scientific discovery process. By easily creating new examples with different missing variables, our benchmarks also test the robustness of models' parametric knowledge and their propositional reasoning between variables.

Paper Type: Long

Research Area: Machine Learning for NLP

Research Area Keywords: Causality, Discovery, LLMs, Reasoning

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 4647