A Truthful and Accurate Forecasting Competition Mechanism on Bayesian Network Structured Events

Chun Lau, Daniel Schoepflin, David Pennock

Abstract:

We provide the first forecasting competition mechanism that is both truthful and accurate for arbitrarily correlated events. Previous work describes mechanisms that either work only for independent events or block-correlated events or that do not ensure accuracy.
Our mechanism works for any event structure that can be represented as a decomposable Bayesian network, an assumption that is without loss of generality, since any joint distribution can be represented as a Bayes net (BN), and any BN can be made decomposable.
We generalize the Event-Lotteries Forecaster Selection Mechanism (ELF)
which works
for independent events [Witkowski et al., 2018, 2023].
We first show that ELF is truthful for any two events, regardless of correlation. We next show that the counterexample of Witkowski et al. [Witkowski et al., 2018, 2023] for correlated events is circumvented if forecasters can update their beliefs over time.
We then create a new counterexample of only three correlated events showing that ELF fails even if forecasters can update. Finally, we design a new competition mechanism called Bayesian network ELF (BNELF) that is truthful for any structure of correlation represented as a decomposable BN (e.g., a tree or join tree).
BNELF divides the competition into two subgames. By asking forecasters to provide probabilities for each event $E$ in one subgame conditioned on all possible combinations of outcomes of $E$'s children in the BN in the second subgame, the events in each subgame become conditionally independent.
Finally, BNELF flips a coin; forecasters play ELF randomly in one of the two subgames. We show that, given enough events that are at least minimally uncertain after conditioning,
BNELF is guaranteed to accurately converge to identify the best forecaster.