Adapting Multiverse Analysis for Prediction: A Decision-Maker’s Dashboard

Fifi Ding, Galit Shmueli, Travis Greene, Sofie Goethals

Track: Main track

Published Or Accepted: false

Keywords: multiverse analysis, specification curve analysis, Rashomon effect, regression tree, dashboard, prediction, decision-makers, high-stakes, machine learning fairness

TL;DR: An interactive dashboard that is powered by regression trees, which reveals how predictions change under different pipeline modeling choices

Abstract: Algorithmic decision-making systems increasingly guide consequential judgments in domains such as criminal justice, credit scoring, and healthcare. Yet, their legitimacy is undermined by predictive inconsistency. When the same person receives different risk scores depending on which analytical method is used, even when all methods are reasonable, individuals with highly variable predictions face unfair treatment based on arbitrary methodological choices. While prior research conceptualized this multiplicity issue and proposed multiverse analysis (specification curves) to expose variability, existing tools remain inaccessible to non-technical stakeholders who rely on algorithmic predictions in high-stakes decisions. This study introduces an interactive multiverse visualization dashboard that translates complex analytical variability into interpretable insights for decision-makers, using regression trees in the backend. Unlike conventional explainability tools that focus on how one model treats different individuals, our framework explores how one individual can yield multiple algorithmic outcomes across plausible analytical pipelines. The dashboard integrates three innovations: (1) interactive specification curves adapted for probabilistic outcomes, (2) a regression tree-driven engine to help identify which modeling decisions most influence prediction variability, and (3) profile comparison that enables real-time visual counterfactual exploration of feature variations. Through these features, we provide an evidence-based support tool to inform decision-makers in various high-stake domains.

Submission Number: 9