Neutralizing Self-Selection Bias in Sampling for Sortition

Bailey Flanigan, Paul Goelz, Ariel D. Procaccia, Anupam Gupta

Abstract:

Sortition is a political system in which decisions are made by panels of randomly
selected citizens. The process for selecting a sortition panel is traditionally thought
of as uniform sampling without replacement, which has strong fairness properties.
In practice, however, sampling without replacement is not possible since only a
fraction of agents is willing to participate in a panel when invited, and different
demographic groups participate at different rates. In order to still produce panels
whose composition resembles that of the population, we develop a sampling
algorithm that restores close-to-equal representation probabilities for all agents
while satisfying meaningful demographic quotas. As part of its input, our algorithm
requires probabilities indicating how likely each volunteer in the pool was to
participate. Since these participation probabilities are not directly observable, we
show how to learn them, and demonstrate our approach using data on a real sortition
panel combined with information on the general population in the form of publicly
available survey data.