Generative Social Choice: The Next Generation

Niclas Boehmer, Sara Fish, Ariel D. Procaccia

Abstract: A key task in certain democratic processes is to produce a concise slate of statements that proportionally represents the full spectrum of user opinions. This task is similar to committee elections, but unlike traditional settings, the candidate set comprises all possible statements of varying lengths, and so it can only be accessed through specific queries. Combining social choice and large language models, prior work has approached this challenge through a framework of generative social choice. We extend the framework in two fundamental ways, providing theoretical guarantees even in the face of approximately optimal queries and a budget limit on the overall length of the slate. Using GPT-4o to implement queries, we showcase our approach on datasets related to city improvement measures and drug reviews, demonstrating its effectiveness in generating representative slates from unstructured user opinions.

Lay Summary:

On many online platforms, users share their opinions, ranging from product reviews on Amazon to public policy discussions on Pol.is. But when hundreds or thousands of people weigh in, it becomes impossible for any one person to read everything.

We introduce the Proportional Slate Engine (PROSE), a system that creates a slate of statements from large, messy collections of opinions. Instead of repeating popular comments, PROSE uses a language model (like ChatGPT) to write and assess new statements that synthesize the beliefs of groups of users. Inspired by ideas from voting theory, PROSE aims for proportional fairness: each viewpoint receives space in the summary proportional to its popularity.

A key challenge is ensuring both clarity and fairness. To keep things simple, PROSE respects a word limit, so the summary doesn't become too long. To ensure fairness, we provide mathematical guarantees: even if the language model makes small mistakes, the summary still proportionally reflects what people believe.

We tested PROSE on real-world data like drug reviews and city improvement debates, and found it to be a powerful tool for summarizing public input in a readable and fair way.

Link To Code: https://github.com/sara-fish/gen-soc-choice-next-gen

Primary Area: Theory->Game Theory

Keywords: Social choice, large language models, committee elections, democratic processes, proportional fairness

Flagged For Ethics Review: true

Submission Number: 8232