Prequential Evidence Pruning: Information-Theoretic Edge Selection for Ordering-Based Causal Discovery

ICLR 2026 Conference Submission22354 Authors

Keywords: Causal Discovery, Ordering-based Methods, Information Theory, Minimum Description Length, Conditional Mutual Information, Prequential Scoring, Context-aware Pruning

TL;DR: PEP is a plug‑in pruning module for ordering‑based causal discovery: it keeps an edge only if its out‑of‑sample (prequential) log‑likelihood gain exceeds a computed MDL gate, yielding consistent gains across multiple backbones.

Abstract: Ordering-based causal discovery reduces structure learning to parent selection under a candidate order, yet its pruning stage remains the primary bottleneck: widely used procedures rely on marginal, additivity-constrained tests and tuned thresholds, which fail to capture non-additive interactions and compromise reproducibility. We introduce *Prequential Evidence Pruning (PEP)*, a framework that reframes pruning as a local cost-benefit analysis grounded in information theory. For each candidate edge, PEP computes a prequential (out-of-fold) log-evidence gain by evaluating the child's predictive density in the context of its current co-parents, and retains the edge only when this gain exceeds a computed Minimum Description Length (MDL) code-length penalty that adapts to sample size, the number of admissible parents, and the set size. Theoretically, the population target of the evidence gain equals conditional mutual information (CMI); the statistic is stable under bounded log-loss regret of the predictive component; and prequential scoring yields finite-sample concentration. Empirically, instantiating PEP with a pre-trained tabular model that provides calibrated, zero-shot predictive densities yields consistent improvements across diverse ordering backbones and datasets, including stress tests under misspecification. PEP thus replaces fragile heuristics with a principled, auditable rule, elevating the pruning stage of ordering-based discovery from marginal testing to context-aware evidence maximization.

Primary Area: causal reasoning

Submission Number: 22354