Revisiting the Predictability of Performative, Social Events

Juan Carlos Perdomo

TL;DR: we shed new light on old debates regarding the predictability of social events and performative outcomes

Abstract: Social predictions do not passively describe the future; they actively shape it. They inform actions and change individual expectations in ways that influence the likelihood of the predicted outcome. Given these dynamics, to what extent can social events be predicted? This question was discussed throughout the 20th century by authors like Merton, Morgenstern, Simon, and others who considered it a central issue in social science methodology. In this work, we provide a modern answer to this old problem. Using recent ideas from performative prediction and outcome indistinguishability, we establish that one can always efficiently predict social events accurately, regardless of how predictions influence data. While achievable, we also show that these predictions are often undesirable, highlighting the limitations of previous desiderata. We end with a discussion of various avenues forward.

Lay Summary:

When we use algorithms to make predictions about people, these predictions don’t just passively forecast the future: they actively shape it. For instance, predicting that someone is at high risk of a heart attack influences their lifestyle choices and medical treatments in ways that shape their future health outcomes. Once recognized, we see these feedback loops between algorithms and society everywhere, from traffic predictions to stock markets and politics.

Given that predictions are “performative” and shape the data we see, is it possible to reliably predict social events? In this paper, I develop simple procedures that can efficiently produce valid forecasts of social events, no matter the underlying feedback loops. This result places a common issue in social science methodology on formal mathematical footing. It also challenges traditional notions of validity when forecasting social events and motivates further questions around what makes a prediction truly “valuable” if predictions shape social outcomes.

Primary Area: Theory->Learning Theory

Keywords: performative prediction, online learning, multicalibration, outcome indistinguishability

Submission Number: 8427